Mobile information display platforms

ABSTRACT

An advertising system, comprising: a two-wheeled vehicle; a display mounted on the vehicle to show advertisements; and a processor coupled to the display, wherein the processor captures data associated with showing an advertisement on the display, wherein the data includes one or more of the following: advertising parameters from the advertiser; mobile location; mobile distance from a landmark; advertising categories; mobile location demographics; pricing of advertised goods or services; time and date; speed and direction of the advertising display; traffic characteristics associated with the display; views of the display and advertising budget characteristics.

BACKGROUND

The present invention relates to moving information display devices foradvertising.

SUMMARY

In one aspect, a mobile advertising platform includes a display mountedon a moving object such as a person or a vehicle to show advertisements;and a processor coupled to the display, wherein the processor capturesdata associated with showing an advertisement on the display, whereinthe data includes one or more of the following: advertising parametersfrom the advertiser; mobile location; mobile distance from a landmark;advertising categories; mobile location demographics; pricing ofadvertised goods or services; time and date; speed and direction of theadvertising display; traffic characteristics associated with thedisplay; views of the display and advertising budget characteristics.

In another aspect, a two-wheeled vehicle includes a display mounted onthe vehicle to show advertisements; and a processor coupled to thedisplay, wherein the processor captures data associated with showing anadvertisement on the display, wherein the data includes one or more ofthe following: advertising parameters from the advertiser; mobilelocation; mobile distance from a landmark; advertising categories;mobile location demographics; pricing of advertised goods or services;time and date; speed and direction of the advertising display; trafficcharacteristics associated with the display; views of the display andadvertising budget characteristics.

In a further aspect, a method to correlate consumer purchase with theadvertisements shown on the display includes an app that records aperiodic transmission of advertising vehicle ID over Bluetooth or WiFi,and matches the consumer purchases to the ads shown by the advertisingvehicle ID(s) encountered on a weekly or monthly basis, for example.Purchases of items shown on the displays of nearly advertising vehiclesreflect the effectiveness of the ad, providing significant feedback datafor advertisers. In another aspect, cell phone location data captured bycell towers are correlated with the positions of the advertisingvehicles, and exposure of consumers to the ads can be determined, andconsumer purchases can then be matched against the consumer's exposureto the ad based on proximity of the consumer to the ad vehicle. In yetanother aspect, cell phone location data captured by WiFi locationdatabases, and the location data is correlated with the positions of theadvertising vehicles, and exposure of consumers to the ads can bedetermined, and consumer purchases can then be matched against theconsumer's exposure to the ad based on proximity of the consumer to thead vehicle.

The advertising display has a number of sensors such as IoT (internet ofthings) sensors that can share data with other vehicles and that cancommunicate with the cloud to provide intelligent handling of thevehicle and ads thereon.

In a further aspect, a method for transporting a ride-sharer includes:receiving a pick up request by the ride-sharer; collecting data onpurchasing interests of the ride-sharer; collecting data on purchasinginterests of people on a path to pick up the ride-sharer; displayingadvertisements on a vehicle exterior customized to interests of peoplein each traffic block on the path; adjusting the path to maximizeadvertising earning from the path; and displaying ads inside the vehiclerelevant to the ride-sharer.

In yet another aspect, a method for transporting a ride-sharer includes:receiving a pick up request by the ride-sharer; collecting data onpurchasing interests of the ride-sharer; predicting one or more productsinterested by the ride-sharer and selecting in advance of pick up one ormore offered products; displaying offered products in a space in thevehicle to the ride sharer during the trip; and displaying ads insidethe vehicle relevant to the ride-sharer.

In another aspect, a method for detecting attentiveness includes placinga camera near a consumer, the camera having a wireless radio tocommunicate with a processor; sending a radio signal toward the consumerand detecting a heart rate from the consumer based on a reflected radiosignal; capturing images of the consumer; and detecting attentiveness tothe advertisement based on camera images and the detected heart rate.

In yet another aspect, a method for showing ads includes using a trainedneural network to make driving decisions for an autonomous car; whereinthe data includes one or more of the following: advertising parametersfrom the advertiser; mobile location; mobile distance from a landmark;advertising categories; mobile location demographics; pricing ofadvertised goods or services; time and date; speed and direction of theadvertising display; traffic characteristics associated with thedisplay; views of the display and advertising budget characteristics.

In another aspect, a method for cost-effective navigation of a vehiclein a metropolitan (metro) area, includes capturing images from aplurality of cameras in the vehicle; recognizing objects from the imagesusing one or more neural networks; providing an accelerometer to performposition determination with dead-reckoning; providing positioncoordinates from a global positioning system; receiving positioningcoordinates from a low latency cellular or wife transceiver positionedat a known position; and generating a travel path for the vehicle in themetro area to optimize advertising revenue based on one or more of thefollowing: advertising parameters from the advertiser; mobile location;mobile distance from a landmark; advertising categories; mobile locationdemographics; pricing of advertised goods or services; time and date;speed and direction of the advertising display; traffic characteristicsassociated with the display; views of the display and advertising budgetcharacteristics.

Advantages may include one or more of the following. The system andmethodology supports advertising via movable vehicles such as cars andfleet vehicles including taxis and rental cars and autonomous vehiclesthat leverage the unique nature of these vehicles to provide highlytargeted and ubiquitous advertising opportunities. Owners of vehiclessuch as privately owned automobiles and fleet vehicle owners can derivecompensation in return for making their vehicles available foradvertisements. The system leverages the unique data available invehicles such as automobiles to provide and track highly targetedadvertising opportunities. Such data may include, for example, presentlocation, present speed, average speed, typical routes traveled, andtypical vehicle location at various times during the typical week. Thesystem leverages data which may be available externally in combinationwith specific vehicle data to further target advertising. For example,external databases containing typical traffic volumes on various roadsand highways can be combined with typical driving routes obtained fromspecific vehicles to determine the typical exposure that each suchvehicle has to “eyeballs” during a typical day. Advertisers may interactwith a centralized system to make targeted advertising purchases basedon aggregate or individual vehicle data available in a centralizeddatabase. Such aggregate data may be collected based on aggregatinginformation associated with individual vehicles such as that describedabove. In one embodiment of the present invention, this data iswirelessly transmitted and uploaded from the vehicle to a centralizedsystem for processing and aggregation. Further, in a preferredembodiment of the present invention, the functionality for allowingadvertisers to make advertisement purchases is via a web based interfacecommunicating with a centralized system and database. Vehicle owners mayinteract with a centralized system to make their vehicles available foradvertising and determine actual or likely compensation to be providedin return for making the vehicle available for advertising. In apreferred embodiment of the present invention, the functionality forallowing vehicle owners to make their vehicles available is via a webbased interface communicating with a centralized system and database.The advertising display may be located within or on the outside of avehicle and which functions to implement one or more of the following:receiving advertising data from a centralized system, displayingadvertisements, obtaining vehicle data such as location data and speeddata from the vehicle, and/or reporting such vehicle data to acentralized system. A home appliance or other item present in a home andwhereby such functionality allows for communication with a centralsystem which serves ads to the fixed platform via the said functionalitybased on various targeting characteristics such as location, type ofdevice, demographic data provided by the user of the home appliance orother item and external databases. The system has a centralized systeminteracting with one or more databases and a plurality of dedicateddevices located in vehicles or in a fixed location which communicatewith the central system to receive ads for display and report vehicle orfixed location data to the centralized system. In a preferredembodiment, the centralized system functions to (i) interact withadvertisers so that advertisers can make ad buys; (ii) interact withvehicle owners so that vehicle owners can sign up to make their vehiclesavailable for advertising and provide data associated with their vehicleand other demographic data in connection therewith; (iii) receivevehicle data from vehicles to populate one or more databases withindividual and aggregate information which can be used to price andallocate available advertising opportunities and for other purposes;(iv) transmit advertising data to vehicles via the aforementioneddedicated devices associated with each vehicle; and (v) calculate,reconcile, report and/or process payments which may be due fromadvertisers and which may be payable to vehicle owners based on adsserved or some other agreed to basis. From the advertiser's perspectiveand according to one embodiment of the present invention, advertisersmay log into a web based application which communicates with thecentralized system so that the advertisers can search for, view andpurchase advertising opportunities. By way of example, an advertiser maysearch for an ad buy in the metro Washington, D.C. area where at least50 vehicles are available and wherein each of these vehicles is expectedto be seen by at least 8000 other vehicles during a typical weekday.Once those search results are presented, the advertiser may view thepricing associated with these opportunities and then make an adpurchase. The advertiser could select a start date and end date for thecampaign and during that time frame, the centralized system will servethe desired ads to the selected number of vehicles via the dedicateddevices located in or on the vehicles. The advertiser may furtherconfigure the campaign by selecting specifics such as the type ofvehicles (e.g. SUV, truck, economy car), the specific expected locationsof the vehicle (e.g. typically driving on Rt. 7 weekday mornings,typically parking in Main Street mall on weekend afternoons), times forads to be displayed (e.g. only during the hours from 10-2) and otherselections based on vehicle data obtained from either the vehicle owneras provided to the centralized system (at sign up and/or at a latertime) or obtained though the dedicated device in vehicles periodicallyreporting driving data to the centralized system which is stored inassociated databases. From the vehicle owner perspective, the vehicleowner may access a website/web based interface to sign up for theadvertising service and make his or her vehicle available foradvertising. At the time of sign up, the owner may provide demographicdata possibly including banking information which allows the service tomake payments to the owner periodically. The owner may also provideother data such as make and model of the vehicle, driving habits,locations within or on the vehicle available for the dedicated device tobe located, typical routes driven, where the vehicle is typicalgaraged/parked, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows an exemplary smart vehicle with advertisement display(s)thereon;

FIG. 1B shows an exemplary motorcycle with advertisement display(s)thereon;

FIG. 1C shows various exemplary person mounted ad displays;

FIG. 1D shows an exemplary server based advertisement display controlsystem;

FIG. 2A shows an exemplary smart car electronic system;

FIG. 2B illustrates another exemplary car electronic system;

FIG. 2C illustrates an exemplary gesture control sub-system in thesystem of FIGS. 2A-2B;

FIG. 3-4 show an exemplary processes carried out by the car electronicsystem;

FIG. 5 shows exemplary gesture control of the car;

FIGS. 6A-6C show exemplary obstacles that may be encountered byvehicles;

FIGS. 7A-7H illustrate an exemplary process to fuse data for 3D modelsused for car navigation;

FIGS. 8A-8F show exemplary detection of objects outside of the vehicleand guidance on their handling;

FIGS. 9A-9B show exemplary systems for capturing navigation data andusing such data for smart vehicles;

FIG. 10 shows an exemplary group of cars following flock controlbehavior to display sequenced ads as a collective;

FIG. 11 illustrates a typical network environment in which the systems,methods for cloud based vehicle behavior capturing and monitoring;

FIG. 12 is a diagram illustrating generally, a portion of vehicle alonewith possible locations of sensors, cameras, among others;

FIG. 13 is a diagram illustrating generally, possible locations ofsensors, cameras, and/or other technologies;

FIG. 14 is a sequence diagram illustrates generally, operationsperformed by the system as described in the FIG. 11;

FIG. 15 is a diagram illustrates generally, an overview system that mayallow drivers to obtain action recommendations based on the vehiclebehavior parameters, according to embodiments disclosed herein;

FIG. 16 is a diagram that illustrates generally, an overview ofpreferences matching by the server according to embodiments disclosedherein;

FIG. 17 is a flow chart illustrates generally, a method for selectivelyproviding insurance information to a service provider, according toembodiments as disclosed herein;

FIG. 18 is a diagram illustrates generally, an exemplary system thatcustomizes insurance rates to correspond to behavior driver, accordingto embodiments as disclosed herein;

FIG. 19 is a diagram illustrates generally an insurance rate adjustmentcomponent that further includes an analyzer component, according toembodiments as disclosed herein;

FIG. 20 illustrates generally, a method for customizing insurance ratesof a driver, according to embodiments as described herein;

FIG. 21 illustrates generally, a method for presenting informationrelated to a real-time insurance rate, according to embodiments asdescribed herein;

FIG. 22 is diagram illustrates generally, a method for installation of areal-time insurance system, according to embodiments disclosed herein;

FIG. 23 is a diagram illustrates generally, a method for gatheringinformation from an on-board monitoring system employed in a real-timeinsurance system, according to embodiments as disclosed herein;

FIG. 24 is a diagram illustrates generally, a method mounting cameras tocapture traffic information, according to embodiments as disclosedherein;

FIG. 25 is a diagram illustrates generally, a method mounting cameras tocapture vehicle behavior, according to embodiments as disclosed herein;and

FIG. 26 is a diagram illustrates generally, a first vehicle programcommunicating with a second vehicle program through an Inter-VehicleCommunication, according to embodiments as disclosed herein.

DESCRIPTION

FIG. 1A shows an exemplary environmentally friendly vehicle such as acar 1 with a passenger compartment 2 and a central engine compartment 3behind passenger compartment 2 with a front window 14 and one or moreside windows and a rear window. Although the engine compartment 3 isshown as a rear-engine, the engine compartment 3 can also be a frontengine compartment. The engine can be all electric engine, hydrogenengine, hybrid engine, or an ultra low emission gas engine. A frame 4 ofthe car 1 supports a roof 5 which can be a sun roof that can expose thepassenger compartment 2 in an open position and can cover the passengerwhen closed. To support the sun roof, the frame 4 provides two verticalposts 6 facing each other on opposite sides of car 1, at the boundarybetween passenger compartment 2 and engine compartment 3. When sun roof5 is in the closed position, roof members 7 and 8 are substantiallyhorizontal, substantially coplanar, and positioned seamlessly one behindthe other. The car contains a cooling system that minimizes the weightand power consumption of conventional air conditioning system for thecar 1. The car 1 also includes one or more advertising displays 136 aspart of a mobile advertising system 130. Typically each display 136 hasa plurality of LED panels driven by power drivers and controlled by aprocessor that communicates over wireless networks such as WiFi orcellular. Preferably the displays 136 have its own energy source so thatads can be shown even if the vehicle is powered down. In one embodiment,the display 136 has motorized positioners that can adjust the positionof the display, the tilt and pitch, among others, so that the view isoptimized for target viewers.

FIG. 1B shows an exemplary two-wheel embodiment where the display 136surrounds a delivery box. The driver can put food or items to bedelivered into the delivery box and proceeds to the destination whileads are shown on at least 3 sides of the delivery box. FIG. 1C showshuman mounted ad units with displays 136 that are secured to the body ofthe human showing the ads.

As shown in FIG. 1D, the system 130 is controlled by a remoteadvertising server 120 which may execute computer readable instructionmodules for performing functionality discussed herein. These modules maybe collectively referred to as advertising server modules 122. Theadvertising server 120 may also store programs, data, and advertisingcontent on server storage 124. The mobile advertising system 130 maycommunicate with the advertising server 120 over the network 20 using adata radio 132 associated with the mobile advertising system 130. Itshould be appreciated that the network 20 connecting the advertiser 110with the advertising server 120 may be the same, similar, entirelydifferent, or partially different from the network 20 connecting theadvertising server 120 to the mobile advertising system 130. The network20 may be the Internet, a local network, metro network, wide-areanetwork, 3G/4G/5G/6G mobile communications network, wireless, wired,optical, electrical, radio, packet-based, connection-based, telephoneline, other network, or any combination thereof. A position locator 134may be associated with the mobile advertising system 130 to supportdetermining location and other characteristics of the mobile advertisingsystem 130. Alternatively, the location can be retrieved from the carpositioning system. The locator may be a global positioning system(GPS), a global navigation satellite system (GNSS), a local positioningsystem, a mobile communication assisted positioning system, LORAN, orany other positioning or location system. The display device 136 andaudio device 138 may be associated with the mobile advertising system130 for displaying advertising content. It should be appreciated thatreferences herein to displaying content includes playing, projecting,decoding, broadcasting, presenting, or otherwise making the contentavailable to be seen and/or heard by one or more individuals. The mobileadvertising system 130 and display device 136 may include indoor oroutdoor display systems. The display device 136 may be mounted on apeople mover such as a taxi cab, bus, truck, blimp, balloon, train, orso forth. Mobile storage 140 at the mobile advertising system 130 may beused for storing data, or programs. This may include any advertisingcontent to be displayed. This may also include machine executablesoftware code referred to collectively as mobile advertising modules142. The advertising server 120 and associated advertising servermodules 122 can support scoring and fitness metric functions andalgorithms to determine at what time and at what mobile locationadvertising content may be displayed. The advertising server 120 andassociated advertising server modules 122 may support interfaces foraccess by web browsers, computer applications, user application, mobileapplications, or any other access interface for users or machines. Theadvertising server 120 can allow advertisers 110 to register, login,make payment and upload their advertisement media content. Relatedfunctions may include registering advertiser 110 (user name, password,contact info, name, email, address). After registering an advertiser 110the system may notify a system administrator to review the registration.The mobile advertising system 130 may also support advertiser 110 to addfunds. The advertiser 110 may use an online merchant solution and/ordirect credit card payment to fund their account used to pay foradvertising. The mobile advertising system 130 may also supportadvertiser 110 to upload media content by: prompting for media type(image, video, audio, etc.), prompting file location/path, promptingadvertisement parameters (e.g. total number of minutes/hours to displayadvertisement, desired times, locations, demographics, etc.). This maybe validated against available funds balance in advertiser's account.The desired location for advertising may be given by address, zip code,longitude/latitude, or any other geographic locator. A range may bespecified such as a radius or number of surrounding zip codes,neighborhoods, or any other geographic demarcation unit. Dates and timesmay also be specified for when to display, or not display, advertisementcontent. Advertising fees may be a function of the specificity ofadvertising parameters such as time of day, location, and so forth. Forexample, an advertisement targeting rush-hour traffic may incur a higherfee than an advertisement that can be displayed at any time of day.Similarly, advertisers may submit advertisement content with fewconstraints for reduced fees. The mobile advertising system 130 canvalidate the address and calculate latitude and longitude of the targetlocation. The mobile advertising system 130 can verify, or cause to beverified, the advertising content and parameters through a qualityassurance (QA) review. The QA review can validate the appropriatenessand the legitimacy of the advertising content. If the content passes theQA process, it may be published to “media pool” or “content pool” asready for publication or web-service access. The mobile advertisingsystem 130 can support retrieval of advertising media statistics ordisplay information to allow advertisers 110 to reviewstatistics/metrics about advertisement media deployment. The advertiser110 may select a desired advertisement that they previously uploaded.The mobile advertising system 130 may present a map displaying locationswhere the advertisement was displayed. Such a map may have push-pinsrendered in locations where advertisement media has been displayed. Suchpush-pins may be selected (or clicked) to show details such as:advertisement display location (e.g. begin and ending latitude/longitudecoordinates), advertisement display start and end time, advertisementdisplay duration, and any other related details. The mobile advertisingsystem 130 may include dedicated or custom advertisement hardware. Thishardware may communicate with the advertising server 120 to authenticateitself using a hardware serial number. The mobile system may alsoprovide location information to the server. It should be appreciatedthat this location information may include data indicating the location,speed, direction, and other parameters associated with the mobileadvertising system 130. The mobile advertising system 130 can continueto execute previously received instructions when communication with theadvertising server 120 is unavailable. For example, communicationbreakdowns may occur due to network outage, coverage limitations (e.g.when passing through a tunnel), or due to weather related issues. Themobile advertising system 130 can locally log any autonomous advertisingactivities during such periods. The log may then be transmitted to theadvertising server 120 as a batch when connectivity is re-established.The advertising server 120 can then determine if those advertisementsstill qualify for billing against the advertiser 110. Similarly, themobile advertising system 130 can continue to use a last known locationposition to continue regular operation for example when there istemporarily no GPS satellite signal.

In one embodiment, the vehicle can query nearby smart phones through avehicle to vehicle (V2V) or V2C communication protocol that facilitateidentifying peers based upon encoded signals during peer discovery in apeer to peer network. The system can be WiFi or cellular based such asthe Proximity Services via LTE Device Broadcast, among others. In oneembodiment, the identification of peers based upon encoded signalsduring peer discovery in a peer to peer network can be done. Forexample, direct signaling that partitions a time-frequency resource intoa number of segments can be utilized to communicate an identifier withina peer discovery interval; thus, a particular segment selected fortransmission can signal a portion of the identifier, while a remaindercan be signaled based upon tones communicated within the selectedsegment. Moreover, a subset of symbols within the resource can bereserved (e.g., unused) to enable identifying and/or correcting timingoffset. Further, signaling can be effectuated over a plurality of peerdiscovery intervals such that partial identifiers communicated duringeach of the peer discovery intervals can be linked (e.g., based uponoverlapping bits and/or bloom filter information). The method caninclude transmitting a first partial identifier during a first peerdiscovery interval. Also, the method can comprise transmitting a secondpartial identifier during a second peer discovery interval. Further, themethod can include generating bloom filter information based upon thecombination of the first partial identifier and the second partialidentifier. Moreover, the method can comprise transmitting the bloomfilter information to enable a peer to link the first partial identifierand the second partial identifier. Another embodiment communicates usingLTE Direct, a device-to-device technology that enables discoveringthousands of devices and their services in the proximity of ^(˜)500 m,in a privacy sensitive and battery efficient way. This allows thediscovery to be “Always ON” and autonomous, without drasticallyaffecting the device battery life. LTE Direct uses radio signals—called‘expressions’—which can be private and discreet (targeted securely forcertain audiences only) or public (transmitted so that any applicationcan receive them). Public expressions are a common language available toany application to discover each other, and this is the door to consumerutility and adoption. Public expressions exponentially expand the fieldof value. For example, vehicles that share same driving segments canbroadcast expressions indicating their path(s). The system detectsvehicles in the same segment as part of the proximity services forcapturing and sharing crowd-sourced navigation data. Public expressionscombine all applications—all value—into one single network, therebyexpanding the utility of the system. Using the expressions, theprocessor can query to identify all nearby vehicles and smart phones andsuch information can later be used to match up against purchases by theconsumer (once permission is granted to access credit card to determinethe effectiveness of the ads.

An advanced scoring algorithm can determine the relevance and order inwhich advertisements are displayed on each mobile unit. Some of theinputs to the scoring algorithm may include distance, such as theproximity of the advertisement panel to the actual target businesslocation. One of the main objectives of the scoring algorithm includesseeking to maximize advertising revenue by displaying advertisementsaccording to a more relevant or optimized schedule or pattern. Anothermain objective of the scoring algorithm includes seeking to maximizegeographic and temporal coverage.

Other inputs to the scoring may include categories, such as havingadvertisements classified into different business types such asrestaurant, beauty salon, department store, etc. The location of themobile advertising display panel may also be compared against aregion/zone business catalog information database that ranks each zoneby business type. A similarity score index may be run against theadvertisement category and the region zone catalog. Advertisements withhigh similarity index may have higher score than those with lowersimilarity index.

Other inputs to the scoring may include pricing. For example,advertisers may have the option to enter the average dollar amount thata consumer will have to pay to take advantage of the advertisement theyare submitting. This price can be classified into continuous or discretefactor levels (ex: low, medium, high, etc.) that may be compared againstthe publicly available income-per-household census data for eachrespective location advertisement panels are located.

Other inputs to the scoring may include advertiser purchasing power. Forexample, high-paying advertisers may be given a competitiveadvantage/dominance over those that do not spend as much inadvertisement dollar amount.

Other inputs to the scoring may include time and date. For example, thescoring engine may give special relevance to advertisements based ontheir time sensitivity such that an advertisement for “Lunch” will havea higher score before noon that it will have in the afternoon orevening. The time and date used herein may include the time of day,season, day of week, holiday, other time/date information, or anycombination thereof.

Other inputs to the scoring may include direction of travel. Forexample, advertisement mobiles traveling say from airport zone towardsdowntown may play advertisements in the “hotel” category more frequentlywhile those traveling in the opposite direction will show advertisementsfor businesses located within the airport such as duty free shops, carrentals, airlines, etc.

Other inputs to the scoring may include advertiser funds remaining. Forexample, advertisers whose advertisement budgets are getting depletedmay be given less priority than other advertisements.

Other inputs to the scoring may include traffic characteristics. Thesemay be instantaneous, daily, historic, or any other trafficconsideration.

Scoring content and calculating fitness metrics may use the same orsimilar factors or inputs. These factors may be combined throughaddition, weighted addition, adaptive weighted addition, multiplication,vector or matrix multiplication, arithmetically, geometrically,exponentially, by multiplication with a scaling or fading factor orfunction, as inputs to any other numerical or algebraic function, asinputs to time varying or position varying adaptive algorithms, anyother operation(s), or any combination thereof.

Scoring optimization may also seek to match advertisement parameterswith pre-computed variables through the scoring algorithm. Some exampleadvertising parameters may include: target location for display ofadvertisement, coverage, radius, days of week to display advertisement,time of day to display, average asking price of goods/services inadvertisement, total premium paid, industry classification code foradvertiser, total area coverage requested, total amount of premium paidto date, and so forth.

Some example internal (or pre-computed) parameters for a given locationor area may include: age demographics, income per household, size ofhousehold, housing values, job growth, asking price of goods/services inadvertisement, income, social security income, commute duration, homeforeclosure rate, own/rent statistics, race demographics, educationlevel demographics, geographical mobility metrics, average rent, averageutilities expenses, vehicles per household, indicators of specificindustry involvement (medicine, law, technology, education),unemployment rate changes, distance from recreation (beach, mountains,parks), prevalence of local businesses similar to advertising business,and so forth. These parameters may by specified as means, average,medians, modes, or may be otherwise applied to any other statisticaldigestion.

Accordingly, some example computed variables may include: the currentlocation of the mobile advertising system 130, currenttime/date/month/year, distance of mobile advertising system 130 fromtarget location, time difference between current time and desireddisplay time, funds remaining in advertiser's account, direction oftravel (e.g. heading to airport, into town, out of town), and so forth.

Predicted future locations for the mobile advertising system 130 may becomputed from current location, direction of travel, velocity, map data,and traffic data. Such future information may allow better matching ofadvertisements as well as queuing of relevant advertising content.

Advertisers may access information on their advertising content inreal-time or near-real-time. The advertisers may update the content orparameters regarding the display of the content. The advertisers mayalso update, allocate, or reallocate advertising budgets or accountfunds through the advertising server in real-time or near-real-time.

It should be appreciated that the display mechanisms for images, video,audio, and so forth may be used to display local weather, traffic, maps,public service announcements, safety information, game score results,news, and other information. Such display along with the display ofadvertising content can leverage the dynamic media content handing. Suchdisplay may also be location specific and location aware.

The display 136 can show targeted ads to consumers who see the display136. A determination of which targeted advertisement to obtain for auser can come from and/or be based on personal information, personalpreferences, personal traits, and/or information in a user profile. Byway of example, a user profile includes personal information associatedwith a specific user, such as including characteristics, preferences,and/or habits of the user. An electronic device and/or a user agent thenuse the user profile to determine what targeted advertisements areretrieved and displayed to the user. In order to build the user profile,one or more electronic devices monitor and collect data with respect tothe user and/or electronic devices, such as electronic devices thatassist the user in communicating over networks and electronic devicesthat the user owns and/or interacts. By way of example, this dataincludes user behavior on an electronic device, installed clienthardware, installed client software, locally stored client files,information obtained or generated from a user's interaction with anetwork (such as web pages on the internet), email, peripheral devices,servers, other electronic devices, programs that are executing, etc. Theelectronic devices collect user behavior on or with respect to anelectronic device (such as the user's computer), information about theuser, information about the user's computer, and/or information aboutthe computer's and/or user's interaction with the network. Informationcan be gleamed from purchase history, or conversations, or eye gazesalong with biometrics indicating significant interest in a product whilethe ridesharer is in the vehicle, and through the user's mobile app thatcan capture interest.

Target advertising is a type of advertising in which advertisements areprovided to users based on a certain trait or activity of the user, suchas demographics, purchase history, user preferences, user profile,tracking online or mobile web activities (such as websites visited, websearches performed, hyperlinks clicked), user interface (UI) actions,etc. For example, if a user stares at a sport car and the user heartrate goes up, the system infers interest in that particular sport car.Thereafter, the rider is targeted to receive advertisements related toautomobiles in the vehicle. Targeting advertisement is effective sincethe targeted user is more likely to have an interest in the product whencompared with a non-interested, non-targeted, or random user. By way ofexample, a user profile builder monitors user activities and collectsinformation used to create a user profile. The profile builder monitorsthe user's interactions with one or more electronic devices, the user'sinteractions with other software applications executing on electronicdevices, activities performed by the user on external or peripheralelectronic devices, etc. The profile builder collects both contentinformation and context information for the monitored user activitiesand then stores this information. By way of further illustration, thecontent information includes contents of web pages accessed by the user,graphical information, audio/video information, uniform resourcelocators (URLs) visited, searches or queries performed by the user,items purchased over the internet, advertisements viewed or clicked,information on commercial or financial transactions, videos watched,music played, interactions between the user and a user interface of anelectronic device, commands (such as voice and typed commands),hyperlinks clicked or selected, etc. The user profile builder alsogathers and stores information related to the context in which the userperformed activities associated with an electronic device. By way ofexample, such context information includes, but is not limited to, anorder in which the user accessed web pages (user's browser navigation),a frequency or number of times a user navigated to a web location,information regarding the user's response to interactive advertisementsand solicitations, information about a length of time spent by the useron the web pages, information on the time when the user accessed the webpages, etc. As previously stated, the user profile builder also collectscontent and context information associated with the user interactionswith various different applications executing on one or more electronicdevices. For example, the user profile builder monitors and gathers dataon the user's interactions with a web browser, an electronic mail(email) application, a word processor application, a spreadsheetapplication, a database application, a cloud software application,and/or any other software application executing on an electronic device.By way of illustration, the user profile builder collects contentinformation for emails that include one or more of the recipientinformation, sender information, email subject title information, andthe information related to the contents of the email includingattachments. Context information for an email application may includethe time when the user receives emails, time when the user sends emails,subject matter of the emails, frequency of the emails, recipients, etc.

As yet another example, this determination can include informationextracted from a previous purchase of the user. For instance, if theuser purchases a vehicle (such as a motorcycle, a car, a boat, etc.),the user may be interested in also purchasing an ancillary product forthe vehicle. Alternatively, if the user purchases product X, then theuser may also be interested in product Y since other customers thatpurchased product X also purchased product Y. As yet another example,this determination can include information based on a current, previous,or future physical geographical location of the user. For instance, ifthe user plans a camping trip to a national forest, the user may beinterested in purchasing camping and/or sporting gear. Alternatively,the user may have visited a specific type of store, such as onespecializing in particular product and/or service. As yet a furtherexample, this determination can include information from websites thatthe user visited. For instance, if the user visits a website that sellsprinters or performs a web query about different printers to purchase,the user may be interested in purchasing a new printer. As yet anotherexample, this determination includes information of the user's priorinterest or activity in certain sports, travel destinations, clothes orobjects previously purchased, or entertainment preferences. As yetanother example, this determination is based on an age of goods orobjects that the user owns or previously purchased, such as owning anelectronic device that is outdated.

One embodiment performs reverse advertising. For example privateinformation of a first user is analyzed to determine that the first userintends to purchase a first product in the next few months. Privateinformation of a second user is analyzed to determine that the seconduser intends to purchase a second product. The first and second userscommunicate with each other through a video call in the vehicle. Duringthis video call, advertisements are displayed to the users on screens oftheir electronic devices. While the users are talking to each other, anadvertisement for the second product plays on the display of theelectronic device of the first user while simultaneously anadvertisement for the first product plays on the display of theelectronic device of the second user. The advertisements are switchedsuch that the first user sees the advertisement of the second user, andthe second user sees the advertisement of the first user.

One embodiment of the system 130 operates as follows. Advertisingcontent may be received from the advertiser 110 into the advertisingserver 120. The advertising server 120 can store advertising contentreceived from the advertiser 110. The advertising server 120 can receiveadvertising parameters from advertiser 110. The advertising server 120can score and categorize the advertising content received from theadvertiser 110. The advertising server 120 can receive payment from theadvertiser 110. The advertising server 120 can receive locationinformation from one or more mobile advertising systems 130. Thelocation information may include position, direction, velocity, path,predicted path, predicted delay characteristics, and any other inputfrom the location to support scoring and allocating advertisingopportunities. The advertising server 120 may also receive the localtime for each of the mobile advertising systems 130 or the local timesmay be calculated given the location of the mobile advertising system130. These local times may be used in computing the fitness metrics forselecting advertising to be presented at a given mobile advertisingsystem 130. The advertising server 120 can compute a fitness metric foradvertising content associated with one or more mobile advertisingsystems 130. It should be appreciated that, according to one or moreexample embodiments, some or all of the functionality of computing thefitness metric for advertising content may instead be performed at themobile advertising systems 130. For example, the fitness metrics may becomputed in part, or whole, in a distributed fashion by the one or moremobile advertising systems 130, centrally by the advertising server 120,or by some combination thereof. The advertising server 120 can determinepreferred advertising content for one or more mobile advertising systems130. The determination may be based on the computed fitness metrics. Itshould be appreciated that, according to one or more exampleembodiments, some or all of the functionality of determining preferredadvertising content may instead be performed at the mobile advertisingsystems 130. For example, the preferred advertising content may bedetermined in part, or whole, in a distributed fashion by the one ormore mobile advertising systems 130, centrally by the advertising server120, or by some combination thereof. The advertising server 120 maytransmit advertising content to one or more mobile advertising systems130 according to preferred advertising content. The advertising server120 can receive advertising report information from one or more mobileadvertising systems 130. The advertising server 120 can provideadvertising reports to advertisers 110 based upon information received.

During operation, the mobile advertising system 130 may transmitlocation info to the advertising server 120 and receive advertisingand/or advertising parameters from the advertising server 120. Themobile advertising system 130 may store advertising content receivedfrom the advertising server 120 and may display advertising content. Themobile advertising system 130 may report parameters of displayed contentto the advertising server 120.

In one embodiment, the system provides payments to the driver of the caror the owner of the car based on the following: advertising parametersfrom the advertiser; mobile location; mobile distance from a landmark;advertising categories; mobile location demographics; pricing ofadvertised goods or services; time and date; speed and direction of theadvertising display; traffic characteristics associated with theadvertising display; and advertising budget characteristics.

A financial settlement engine serves to settle financial transactionsassociated with the advertising program(s). This includes calculatingand processing payments due from each advertiser based upon agreed toamounts as well as calculating and processing payments due to eachvehicle owner participating in advertising programs, again, at agreed torates. Rates for paying vehicle owners may be based, for example, on amonthly amount regardless of the number of ads served, on a per adbasis, on a per mile driven basis, based upon the type of vehicledriven, the location(s) that the vehicle is typical driven/parked or anyother basis determined by the program operator. Rates for chargingadvertisers may be based, for example, on a fixed monthly amount, on aper ad impression basis, on a custom package of impressions as selectedby an advertiser such as types of vehicles, geographic locations ofvehicles, typical miles driven, etc. or any other basis determined bythe program operator. Financial settlement engine may connect with oneor more financial institutions, gateways or other systems or serviceswhich allows for payments to be made via electronic transfer to and fromvehicle owners and advertisers.

Advertisers desiring to make advertising buys using the service willregister through a web based interface. At the initial advertiserregistration, various information may be collected to set up anadvertiser account. Examples include company name, contact information,expected monthly spend, advertising preferences, bank accountinformation and amount of initial deposit for advertising costs. Thesystem sets up one or more records associated with the advertiseraccount for storage in advertiser database. As the advertiser makesadditional advertising buys, historical information may be obtained andprocessed to, for example, make suggestions for potential ad buys forthe advertiser in the future. This information may be stored inadvertiser database. Advertisers may upload advertisements that theydesire to have served to the cars via WiFi or 5G networks. This ispreferably accomplished under the control of advertiser engine such thatall available ads associated with that advertiser are stored inadvertisements database. Advertisements may be uploaded, stored andserved according to many known digital formats such as those associatedwith still pictures, video and others. Once advertisers have an accountset up, they may make buys of advertising in connection with theadvertising program made available through the systems and methodologiesof the present invention. Based on data acquired from vehicle ownerseither manually through a web interface as described above, aggregateddata can be generated and the various advertising opportunities/packagesmade available through a web based interface to advertisers forselection. For example, aggregated data such as that may be generatedfrom data derived from vehicle owners as described herein, can make thefollowing types of targeted advertising packages available:

Within a particular geographic driving area

A specific number of vehicles

Vehicles of only a certain make, model or year

Vehicles that only drive a certain number of miles each day

Vehicles that are present at a certain location (e.g. shopping mall) fora minimum number of hours per day

Vehicles that only drive a specific route or some portion thereof onspecific days

Vehicles that only drive within a certain average speed range

Vehicles that place their advertising displays only in a certainlocation on their vehicle (e.g. rear window deck, bumper, etc.)

Various other selection criteria could also be used based upon thetargeted data obtained from vehicle owners. In addition to these typesof constraints, advertisers may also control their ad purchases throughother criteria such as total number of ads served, maximum spend oversome period of time (monthly, weekly etc.), days/times for ads to beserved, etc. So, therefore, it can be seen by one of ordinary skill inthe art that the present invention provides a unique opportunity fortargeted advertising according to a wide range of criteria so that adspend can be maximized. Given the web based nature of the system,advertisers can also track success of ads based on desired criteria suchas store traffic, purchases etc. in connection with various ad campaignsand, if desired, change ad and spend constraints as described above overtime to maximize effectiveness.

In some jurisdictions or for other reasons, it may not be desirableand/or legal to display either still images or video in a vehicle whilethat vehicle is moving within traffic. In this case, it is possible toemploy the teachings of the present invention to display advertisementsonly when the vehicle which holds the dedicated display device 136 iseither stopped or driving below a certain speed.

Either or both of GPS or input from vehicle via vehicle input processormay be used to obtain current speed information for the vehicle and GPSmay be used to obtain location data. This data can be used, for example,to ensure that ads are displayed only when speed is at zero (vehiclestopped) or, for example, at a current speed of less than 15 miles perhour. In addition, location data can be used to ensure ads aredisplayed, if desired, only at specific locations or within specificgeographic perimeters. In this way, for example, ads may be displayedonly when the vehicle is parked at one or more mall parking lots in themetro Washington D.C. area. Ads can thus be targeted, perhaps, to storeslocated within that mall and may be served and displayed in real time ornear real time so as to permit targeted ads reflecting current specialsor offers. In this embodiment, it is preferable that display 136 containa battery with back up capability so that it may operate when thevehicle engine is turned off.

Another use case involves the exploitation of either or both of time andgeographic data. In this use case, ads may be selected for display basedon a vehicle being within a designated geographic zone and possible alsoduring a certain time. As such, it is possible, for example, for an adto be served in real time or near real time reflecting an ongoingspecial, offer or other advertisement relative to time and location. Sofor example, a local Italian restaurant could buy ads to be served onlyon Mondays-Fridays at between 11 am and 1 pm and only to vehicles withina 3 mile radius of the geographic location of the restaurant. The adcontent could, in this context, relate to specials available at thatrestaurant at those times.

Another use case is presented from the perspective of the vehicle owner.The flexibility of the system of the present invention as well as thediversity of information available to it allows for unique and highlytargeted ad buys as well as for varying schemes for determining paymentdue to vehicle owners. According to this use case, a vehicle owner maysign up and agree to have ads displayed on his or her vehicle's suchthat differing bases for payment are available. For example, an ownermay receive a fixed monthly amount regardless of how many ads aredisplayed or where or when. Alternatively, the owner may receive paymentbased on the number of miles driven, routes taken, type of vehicle,model of display used or other characteristics.

In this context, the teachings of the present invention whereby highlytargeted ads can be served based on location, temperature, time andvarious other factors can be used to target advertisements which areperiodically displayed in a home environment. Residents of the homes inwhich ads are served may be paid in consideration of accepting theseadvertisements either through an monthly or other payment tied to thenumber or type of ads received or possibly though a reduced/subsidizedcost associated with the device (e.g. refrigerator, washing machine)housing the ad display. In this case, ads could be visual and/or may beaccompanied by sound.

Ads could be selected for display and or purchased by advertisers basedon the unique characteristics of the individual and/or aggregated ads.So for example, all vehicles within a 15 mile radius of a specificrestaurant could receive an offer for that restaurant via a display ontheir refrigerator during certain times of day. Other demographic datasuch as that obtained from the owner at the time of sign up or throughimplication, to provide highly targeted advertising to the desiredaudience. Outside of the home environment, for example, at parkingmeters, ads could be served at specific locations and at specific timesas desired by the advertiser.

FIG. 2A shows a block diagram of an embodiment of an electrical powerand automobile control system that includes passenger protection. Thesystem is controlled by a processor 202. The processor 202 is connectedwith an inertial system (INS) 204 and a global positioning system (GPS)receiver 206 that generate navigation information. The processor 202 isalso connected with a wireless communication device 208 that transmitsand receives digital data as well as being a Doppler radar when desired.The processor 202 drives a display 210 and a speaker 212 for alerting adriver. The processor 202 provides control inputs to the automobile'sbraking and steering systems 220. A power cable 200 carries powerbetween the batteries 100-116 and an electric motor engine (not shown).The power cable 200 also carries power to recharge the batteries 100-116serially or in parallel. The data can be provided to wirelesstransmitters that will wirelessly receive the signal and send the dataon to computer stations. Exemplary protocols that can be used includeCAN-bus, LIN-bus over power line (DC-LIN), and LonWorks power line basedcontrol. In one embodiment, the protocol is compatible with the HomePlugspecifications for home networking technology that connects devices toeach other through the power lines in a home. Many devices have HomePlugbuilt in and to connect them to a network all one has to do is to plugthe device into the wall in a home with other HomePlug devices. In thisway, when the vehicle is recharged by plugging the home power line tothe vehicle connectors, automotive data is automatically synchronizedwith a computer in the home or office. This embodiment includesnavigation systems, the INS 204 and the GPS receiver 206. Alternateembodiments may feature an integrated GPS and INS navigation system orother navigation system. The use of only an INS 204 or only a GPSreceiver 206 as the sole source of navigation information is alsocontemplated. Alternatively, the wireless communication device 208 cantriangulate with two other fixed wireless devices to generate navigationinformation. A biologics sensor 210 captures user biological signals andspeaker/microphone 212 provides both visual and audio situationalawareness information to a driver. Alternate embodiments may featureonly a display 210 or only a speaker 212 as the sole source ofinformation for the driver. Embodiments that interact directly with thebraking and steering systems that provide no audio information to thedriver are also contemplated. The braking and steering systems 220 mayalso be commanded by the processor 202. The processor 202 may commandthat the brakes be applied to prevent collision with a vehicle ahead ormay provide a steering input to prevent the driver from colliding with avehicle. The processor 202 may also issue braking or steering commandsto minimize the damage resulting from a collision as discussed in UnitedStates Patent Application 20080091352, the content of which isincorporated by reference.

FIG. 2B is a simplified block diagram of an example vehicle 700, inaccordance with an embodiment. While the vehicle 700 in FIG. 7 isdescribed as being configured to operate in an autonomous mode, in someembodiments the above methods may be implemented in a vehicle that isnot configured to operate in an autonomous mode. In these embodiments,the vehicle may include fewer and/or different systems and/orcomponents. The sensor system 704 may include a number of sensorsconfigured to sense information about an environment in which thevehicle 700 is located, as well as one or more actuators 736 configuredto modify a position and/or orientation of the sensors. As shown, thesensors of the sensor system include a Global Positioning System (GPS)726, an inertial measurement unit (IMU) 728, a RADAR unit 730, a laserrangefinder and/or LIDAR unit 732, and a camera 734. The sensor system704 may include additional sensors as well, including, for example,sensors that monitor internal systems of the vehicle 700 (e.g., an O2monitor, a fuel gauge, an engine oil temperature, etc.). Other sensorsare possible as well. The GPS 726 may be any sensor configured toestimate a geographic location of the vehicle 700. To this end, the GPS726 may include a transceiver configured to estimate a position of thevehicle 700 with respect to the Earth. The GPS 726 may take other formsas well. The IMU 728 may be any combination of sensors configured tosense position and orientation changes of the vehicle 700 based oninertial acceleration. In some embodiments, the combination of sensorsmay include, for example, accelerometers and gyroscopes. Othercombinations of sensors are possible as well. The RADAR 730 unit may beany sensor configured to sense objects in the environment in which thevehicle 700 is located using radio signals. In some embodiments, inaddition to sensing the objects, the RADAR unit 730 may additionally beconfigured to sense the speed and/or heading of the objects. Similarly,the laser rangefinder or LIDAR unit 732 may be any sensor configured tosense objects in the environment in which the vehicle 700 is locatedusing lasers. In particular, the laser rangefinder or LIDAR unit 732 mayinclude a laser source and/or laser scanner configured to emit a laserand a detector configured to detect reflections of the laser. The laserrangefinder or LIDAR 732 may be configured to operate in a coherent(e.g., using heterodyne detection) or an incoherent detection mode. Inone embodiment, a LIDAR-on-a-chip system steers its electronic beamusing arrays of many small emitters that each put out a signal at aslightly different phase. The new phased array thus forms a syntheticbeam that it can sweep from one extreme to another and back again100,000 times a second. In one embodiment, each antenna, which consistsof a silicon waveguide and five curved grooves etched in silicon, is 3micrometers long, 2.8 μm wide, and 0.22 μm thick. An infrared laser beamis delivered to the antennas through a waveguide. The LIDAR 732 can bepart of a camera 734. The camera 734 may be any camera (e.g., a stillcamera, a video camera, etc.) configured to record three-dimensionalimages of an interior portion of the vehicle 700. To this end, thecamera 734 may be, for example, a depth camera. Alternatively oradditionally, the camera 734 may take any of the forms described abovein connection with the exterior camera 610. In some embodiments, thecamera 734 may comprise multiple cameras, and the multiple cameras maybe positioned in a number of positions on the interior and exterior ofthe vehicle 700. The control system 706 may be configured to controloperation of the vehicle 700 and its components. To this end, thecontrol system 706 may include a steering unit 738, a throttle 740, abrake unit 742, a sensor fusion algorithm 744, a computer vision system746, a navigation or pathing system 748, and an obstacle avoidancesystem 750. The steering unit 738 may be any combination of mechanismsconfigured to adjust the heading of vehicle 700. The throttle 740 may beany combination of mechanisms configured to control the operating speedof the engine/motor 718 and, in turn, the speed of the vehicle 700. Thebrake unit 742 may be any combination of mechanisms configured todecelerate the vehicle 700. For example, the brake unit 742 may usefriction to slow the wheels/tires 724. As another example, the brakeunit 742 may convert the kinetic energy of the wheels/tires 724 toelectric current. The brake unit 742 may take other forms as well. Thesensor fusion algorithm 744 may be an algorithm (or a computer programproduct storing an algorithm) configured to accept data from the sensorsystem 704 as an input. The data may include, for example, datarepresenting information sensed at the sensors of the sensor system 704.The sensor fusion algorithm 744 may include, for example, a Kalmanfilter, a Bayesian network, or another algorithm. The sensor fusionalgorithm 744 may further be configured to provide various assessmentsbased on the data from the sensor system 704, including, for example,evaluations of individual objects and/or features in the environment inwhich the vehicle 700 is located, evaluations of particular situations,and/or evaluations of possible impacts based on particular situations.Other assessments are possible as well.

The computer vision system 746 may be any system configured to processand analyze images captured by the camera 734 in order to identifyobjects and/or features in the environment in which the vehicle 700 islocated, including, for example, traffic signals and obstacles (e.g., inembodiments where the camera 734 includes multiple cameras, including acamera mounted on the exterior of the vehicle 700). To this end, thecomputer vision system 746 may use an object recognition algorithm, aStructure from Motion (SFM) algorithm, video tracking, or other computervision techniques. In some embodiments, the computer vision system 746may additionally be configured to map the environment, track objects,estimate the speed of objects, etc. The navigation/path system 748 maybe any system configured to determine a driving path for the vehicle700. The navigation/path system 748 may additionally be configured toupdate the driving path dynamically while the vehicle 700 is inoperation. In some embodiments, the navigation and path system 748 maybe configured to incorporate data from the sensor fusion algorithm 744,the GPS 726, and one or more predetermined maps so as to determine thedriving path for the vehicle 700. The obstacle avoidance system 750 maybe any system configured to identify, evaluate, and avoid or otherwisenegotiate obstacles in the environment in which the vehicle 700 islocated. The control system 706 may additionally or alternativelyinclude components other than those shown. Peripherals 708 may beconfigured to allow the vehicle 700 to interact with external sensors,other vehicles, and/or a user. To this end, the peripherals 708 mayinclude, for example, a wireless communication system 752, a touchscreen754, a microphone 756, and/or a speaker 758.

The wireless communication system 752 may take any of the formsdescribed above. In one embodiment, it can be the Dedicated Short RangeCommunications (DSRC) which provides the communications-based activesafety systems. DSRC communications take place over a dedicated 75 MHzspectrum band around 5.9 GHz, allocated by the US Federal CommunicationsCommission (FCC) for vehicle safety applications. In contrast to WiFi,DSRC can accommodate an extremely short time in which devices mustrecognize each other and transmit messages to each other. A large numberof these safety applications require response times measured inmilliseconds. DSRC is targeted to operate in a 75 MHz licensed spectrumaround 5.9 GHz, as opposed to IEEE 802.11a that is allowed to utilizeonly the unlicensed portions in the frequency band. DSRC is meant foroutdoor high-speed vehicle (up to 120 mph) applications, as opposed toIEEE 802.11a originally designed for indoor WLAN (walking speed)applications. In IEEE 802.11a, all PHY parameters are optimized for theindoor low-mobility propagation environment. Communications-based activesafety applications use vehicle-to-vehicle (V2V) andvehicle-to-infrastructure (V2I) short-range wireless communications todetect potential hazards in a vehicle's path—even those the driver doesnot see. The connected vehicle provides enhanced awareness atpotentially reduced cost, and offers additional functionality overautonomous sensor systems available on some vehicles today.Communications-based sensor systems provide a low-cost means of enablinghazard detection capability on all vehicle classes, but requiresvehicles and infrastructure to be outfitted with interoperablecommunications capabilities of DSRC or similar Vehicle to Vehiclenetworks.

The car can have a low latency 5G transceiver that communicates to acell tower, and processing resources such as GPU and array processorsnear the cell tower can provide high speed shared compute power to thecar through the 5G network. For example, the 5G network can havemillimeter transceiver such as a low latency ultra-wide-band transceiverin communication with the processor and a remote processor canreceive/send data to the transceiver to offload processing from theprocessor. Such extra power can be useful in AR/VR applications withsurround 8k videos processed as 360 degree videos. The extra power canbe used for road side recognition of objects, and for generating highdefinition maps as the car drives through an area with construction andchanged from the last HD map, for example.

The touchscreen 754 may be used by a user to input commands to thevehicle 700. The microphone 756 may be configured to receive audio(e.g., a voice command or other audio input) from a user of the vehicle700. Similarly, the speakers 758 may be configured to output audio tothe user of the vehicle 700. Still further, while the above descriptionfocused on a vehicle 700 configured to operate in an autonomous mode, inother embodiments the vehicle may not be configured to operate in anautonomous mode. In these embodiments, for example, one or more of thefollowing components may be omitted: the global positioning system 726,the inertial measurement unit 728, the RADAR unit 730, the laserrangefinder or LIDAR unit 732, the actuators 736, the sensor fusionalgorithm 744, the computer vision system 746, the navigation or pathsystem 748, the obstacle avoidance system 750, the wirelesscommunication system 752, the touchscreen 754, the microphone 756, andthe speaker 758.

Gesture Sensor for Advertising Interaction

FIG. 2C shows an exemplary gesture recognition system to provideinteractive advertising with consumers. The system takes advantage ofthe numerous cameras onboard the vehicle for navigation and mappingpurposes, and additionally includes the gesture control feature. System800 includes a pair of cameras 802, 804 coupled to an image-analysissystem 806. Cameras 802, 804 can be any type of camera, includingcameras sensitive across the visible spectrum or, more typically, withenhanced sensitivity to a confined wavelength band (e.g., the infrared(IR) or ultraviolet bands); more generally, the term “camera” hereinrefers to any device (or combination of devices) capable of capturing animage of an object and representing that image in the form of digitaldata. For example, line sensors or line cameras rather than conventionaldevices that capture a two-dimensional (2D) image can be employed. Theterm “light” is used generally to connote any electromagnetic radiation,which may or may not be within the visible spectrum, and may bebroadband (e.g., white light) or narrowband (e.g., a single wavelengthor narrow band of wavelengths).

Cameras 802, 804 are preferably capable of capturing video images (i.e.,successive image frames at a constant rate of at least 15 frames persecond), although no particular frame rate is required. The capabilitiesof cameras 802, 804 are not critical to the invention, and the camerascan vary as to frame rate, image resolution (e.g., pixels per image),color or intensity resolution (e.g., number of bits of intensity dataper pixel), focal length of lenses, depth of field, etc. In general, fora particular application, any cameras capable of focusing on objectswithin a spatial volume of interest can be used. For instance, tocapture motion of the hand of an otherwise stationary person, the volumeof interest might be defined as a cube approximately one meter on aside.

System 800 also includes a pair of light sources 808, 810, which can bedisposed to either side of cameras 802, 804, and controlled byimage-analysis system 806. Light sources 808, 810 can be infrared lightsources of generally conventional design, e.g., infrared light-emittingdiodes (LEDs), and cameras 802, 804 can be sensitive to infrared light.Filters 820, 822 can be placed in front of cameras 802, 804 to filterout visible light so that only infrared light is registered in theimages captured by cameras 802, 804. In some embodiments where theobject of interest is a person's hand or body, use of infrared light canallow the motion-capture system to operate under a broad range oflighting conditions and can avoid various inconveniences or distractionsthat may be associated with directing visible light into the regionwhere the person is moving. However, a particular wavelength or regionof the electromagnetic spectrum is required.

It should be stressed that the foregoing arrangement is representativeand not limiting. For example, lasers or other light sources can be usedinstead of LEDs. For laser setups, additional optics (e.g., a lens ordiffuser) may be employed to widen the laser beam (and make its field ofview similar to that of the cameras). Useful arrangements can alsoinclude short- and wide-angle illuminators for different ranges. Lightsources are typically diffuse rather than specular point sources; forexample, packaged LEDs with light-spreading encapsulation are suitable.

In operation, cameras 802, 804 are oriented toward a region of interest812 in which an object of interest 814 (in this example, a hand) and oneor more background objects 816 can be present. Light sources 808, 810are arranged to illuminate region 812. In some embodiments, one or moreof the light sources 808, 810 and one or more of the cameras 802, 804are disposed below the motion to be detected, e.g., where hand motion isto be detected, beneath the spatial region where that motion takesplace. This is an optimal location because the amount of informationrecorded about the hand is proportional to the number of pixels itoccupies in the camera images, the hand will occupy more pixels when thecamera's angle with respect to the hand's “pointing direction” is asclose to perpendicular as possible. Because it is uncomfortable for auser to orient his palm toward a screen, the optimal positions areeither from the bottom looking up, from the top looking down (whichrequires a bridge) or from the screen bezel looking diagonally up ordiagonally down. In scenarios looking up there is less likelihood ofconfusion with background objects (clutter on the user's desk, forexample) and if it is directly looking up then there is littlelikelihood of confusion with other people out of the field of view (andalso privacy is enhanced by not imaging faces). Image-analysis system806, which can be, e.g., a computer system, can control the operation oflight sources 808, 810 and cameras 802, 804 to capture images of region812. Based on the captured images, image-analysis system 806 determinesthe position and/or motion of object 814.

For example, as a step in determining the position of object 814,image-analysis system 806 can determine which pixels of various imagescaptured by cameras 802, 804 contain portions of object 814. In someembodiments, any pixel in an image can be classified as an “object”pixel or a “background” pixel depending on whether that pixel contains aportion of object 814 or not. With the use of light sources 808, 810,classification of pixels as object or background pixels can be based onthe brightness of the pixel. For example, the distance (rO) between anobject of interest 814 and cameras 802, 804 is expected to be smallerthan the distance (rB) between background object(s) 816 and cameras 802,804. Because the intensity of light from sources 808, 810 decreases as1/r2, object 814 will be more brightly lit than background 816, andpixels containing portions of object 814 (i.e., object pixels) will becorrespondingly brighter than pixels containing portions of background816 (i.e., background pixels). For example, if rB/rO=2, then objectpixels will be approximately four times brighter than background pixels,assuming object 814 and background 816 are similarly reflective of thelight from sources 808, 810, and further assuming that the overallillumination of region 812 (at least within the frequency band capturedby cameras 802, 804) is dominated by light sources 808, 810. Theseassumptions generally hold for suitable choices of cameras 802, 804,light sources 808, 810, filters 810, 812, and objects commonlyencountered. For example, light sources 808, 810 can be infrared LEDscapable of strongly emitting radiation in a narrow frequency band, andfilters 810, 812 can be matched to the frequency band of light sources808, 810. Thus, although a human hand or body, or a heat source or otherobject in the background, may emit some infrared radiation, the responseof cameras 802, 804 can still be dominated by light originating fromsources 808,180 and reflected by object 814 and/or background 816.

In this arrangement, image-analysis system 806 can quickly andaccurately distinguish object pixels from background pixels by applyinga brightness threshold to each pixel. For example, pixel brightness in aCMOS sensor or similar device can be measured on a scale from 0.0 (dark)to 1.0 (fully saturated), with some number of gradations in betweendepending on the sensor design. The brightness encoded by the camerapixels scales standardly (linearly) with the luminance of the object,typically due to the deposited charge or diode voltages. In someembodiments, light sources 808, 810 are bright enough that reflectedlight from an object at distance rO produces a brightness level of 1.0while an object at distance rB=2rO produces a brightness level of 0.25.Object pixels can thus be readily distinguished from background pixelsbased on brightness. Further, edges of the object can also be readilydetected based on differences in brightness between adjacent pixels,allowing the position of the object within each image to be determined.Correlating object positions between images from cameras 802, 804 allowsimage-analysis system 806 to determine the location in 3D space ofobject 814, and analyzing sequences of images allows image-analysissystem 806 to reconstruct 3D motion of object 814 using conventionalmotion algorithms.

In identifying the location of an object in an image according to anembodiment of the present invention, light sources 808, 810 are turnedon. One or more images are captured using cameras 802, 804. In someembodiments, one image from each camera is captured. In otherembodiments, a sequence of images is captured from each camera. Theimages from the two cameras can be closely correlated in time (e.g.,simultaneous to within a few milliseconds) so that correlated imagesfrom the two cameras can be used to determine the 3D location of theobject. A threshold pixel brightness is applied to distinguish objectpixels from background pixels. This can also include identifyinglocations of edges of the object based on transition points betweenbackground and object pixels. In some embodiments, each pixel is firstclassified as either object or background based on whether it exceedsthe threshold brightness cutoff. Once the pixels are classified, edgescan be detected by finding locations where background pixels areadjacent to object pixels. In some embodiments, to avoid noiseartifacts, the regions of background and object pixels on either side ofthe edge may be required to have a certain minimum size (e.g., 2, 4 or 8pixels).

In other embodiments, edges can be detected without first classifyingpixels as object or background. For example, Δβ can be defined as thedifference in brightness between adjacent pixels, and |Δβ| above athreshold can indicate a transition from background to object or fromobject to background between adjacent pixels. (The sign of Δβ canindicate the direction of the transition.) In some instances where theobject's edge is actually in the middle of a pixel, there may be a pixelwith an intermediate value at the boundary. This can be detected, e.g.,by computing two brightness values for a pixel i: βL=(βi+βi−1)/2 andβR=(βi+βi+1)/2, where pixel (i−1) is to the left of pixel i and pixel(i+1) is to the right of pixel i. If pixel i is not near an edge,|βL−βR| will generally be close to zero; if pixel is near an edge, then|βL−βR| will be closer to 1, and a threshold on |βL−βR| can be used todetect edges.

In some instances, one part of an object may partially occlude anotherin an image; for example, in the case of a hand, a finger may partlyocclude the palm or another finger Occlusion edges that occur where onepart of the object partially occludes another can also be detected basedon smaller but distinct changes in brightness once background pixelshave been eliminated.

Detected edges can be used for numerous purposes. For example, aspreviously noted, the edges of the object as viewed by the two camerascan be used to determine an approximate location of the object in 3Dspace. The position of the object in a 2D plane transverse to theoptical axis of the camera can be determined from a single image, andthe offset (parallax) between the position of the object intime-correlated images from two different cameras can be used todetermine the distance to the object if the spacing between the camerasis known.

Further, the position and shape of the object can be determined based onthe locations of its edges in time-correlated images from two differentcameras, and motion (including articulation) of the object can bedetermined from analysis of successive pairs of images. An object'smotion and/or position is reconstructed using small amounts ofinformation. For example, an outline of an object's shape, orsilhouette, as seen from a particular vantage point can be used todefine tangent lines to the object from that vantage point in variousplanes, referred to herein as “slices.” Using as few as two differentvantage points, four (or more) tangent lines from the vantage points tothe object can be obtained in a given slice. From these four (or more)tangent lines, it is possible to determine the position of the object inthe slice and to approximate its cross-section in the slice, e.g., usingone or more ellipses or other simple closed curves. As another example,locations of points on an object's surface in a particular slice can bedetermined directly (e.g., using a time-of-flight camera), and theposition and shape of a cross-section of the object in the slice can beapproximated by fitting an ellipse or other simple closed curve to thepoints. Positions and cross-sections determined for different slices canbe correlated to construct a 3D model of the object, including itsposition and shape. A succession of images can be analyzed using thesame technique to model motion of the object. Motion of a complex objectthat has multiple separately

In some embodiments, the pulsing of light sources 808, 110 can be usedto further enhance contrast between an object of interest andbackground. In particular, the ability to discriminate between relevantand irrelevant (e.g., background) objects in a scene can be compromisedif the scene contains object that themselves emit light or are highlyreflective. This problem can be addressed by setting the camera exposuretime to extraordinarily short periods (e.g., 800 microseconds or less)and pulsing the illumination at very high powers (i.e., 5 to 20 wattsor, in some cases, to higher levels, e.g., 40 watts). This approachincreases the contrast of an object of interest with respect to otherobjects, even those emitting in the same general band. Accordingly,discriminating by brightness under such conditions allows irrelevantobjects to be ignored for purposes of image reconstruction andprocessing. Average power consumption is also reduced; in the case of 20watts for 800 microseconds, the average power consumption is under 80milliwatts. In general, the light sources 808, 110 are operated so as tobe on during the entire camera exposure period, i.e., the pulse width isequal to the exposure time and is coordinated therewith. It is alsopossible to coordinate pulsing of lights 808, 810 for purposes of bycomparing images taken with lights 808, 810 on and images taken withlights 808, 810 off.

Hand-Gesture Interaction with Advertisements

FIG. 5 shows the left arm gesture based interaction process. The processchecks for the raised arm (1002). If the arm is raised (1004) it checksfor the number of fingers raised (1006). The controls for windows areactivated if first four fingers are raised (1008). Further, in someembodiments, the vehicle may determine an extent of the given gesture.For example, if the given gesture is a swipe gesture, the display maydetermine an extent of the swipe (e.g., how long the swipe is in spaceand/or time). The display may then determine an operational parameterbased on the extent. For example, for a greater extent, the display maydetermine a greater operational parameter than for a lesser extent. Theoperational parameter may be, for example, proportional to, orapproximately proportional to, the extent. In these embodiments, whenthe display initiates the function the display may initiate the functionwith the determined operational parameter.

For example, if the ad is about a new car with the latest controlsystem, the advertising hand gesture can allow the consumer to simulateaction in the car. Thus, if the swipe gesture is in a region thatincludes a window, and the swipe gesture in the region that includes thewindow is associated with opening the window, the vehicle may determinean extent of the swipe and further may determine how far to open thewindow based on the extent of the swipe. For instance, the vehicle mayopen the window further for a longer swipe than for a shorter swipe. Asanother example, if the swipe gesture is in a region that includes anair-conditioning vent, and the swipe gesture in the region that includesthe air-conditioning vent is associated with lowering a temperature inthe vehicle, the vehicle may determine an extent of the swipe andfurther may determine how much to lower the temperature in the vehiclebased on the extent of the swipe. For instance, the vehicle may lowerthe temperature further for a longer swipe than for a shorter swipe.Such an extent could be determined for gestures other than a swipegesture as well. For example, if a tap gesture is in a region thatincludes a speaker, and the tap gesture in the region that includes thespeaker is associated with lowering a volume of an audio system, thevehicle may determine an extent of the tap (e.g., how many taps, howlong the tap is held, etc.) and further may determine how much to lowerthe volume of the audio system based on the extent of the tap. Forinstance, the vehicle may lower the volume more for more taps (or alonger tap) than for fewer taps (or a shorter tap). In some embodiments,rather than determining the extent of the gesture and the correspondingoperational parameter and then initiating the function with thedetermined operational parameter, the vehicle may instead continuouslydetermine the extent of the gesture and update the correspondingoperational parameter, and may continuously initiate the function withthe updated operational parameter. In some embodiments, the vehicle mayhave difficulty detecting the given gesture and/or the given region. Forexample, the vehicle may determine that a confidence level of one orboth of the given gesture and the given region is below a predeterminedthreshold. In these embodiments, the vehicle may request an occupant torepeat the given gesture in the given region. When the occupant repeatsthe given gesture in the given region, the vehicle may record additionalthree-dimensional images and may detect the given gesture and the givenregion based on the additional three-dimensional images (and, in somecases, the three-dimensional images previously recorded).

Obstacle Detection and Public Warning Using the Ad Display

The advertising board can improve safety for ride-sharers, and for otherdrivers. For example, the board can display the name of the passenger tobe picked up so that people get into the right vehicle. Further, if thesystem detects a crash ahead, it can warn following cars. In someembodiments, the vehicle identifies obstacles on the road, and thecomputer system may use one or more sensors to sense the obstacles. Forexample, the computer system may use an image-capture device to captureimages of the road and may detect the obstacles by analyzing the imagesfor predetermined colors, shapes, and/or brightness levels indicative ofan obstacle. As another example, the computer system may project LIDARto detect the obstacle. The computer system may estimate the location ofthe obstacle and control the vehicle to avoid the vehicle and yetmaintain a predetermined distance from neighboring vehicles in bothdirections. Other vehicles behind the lead vehicle can then simplyfollow the lead vehicle as part of a flock. The computer system may thencontrol the vehicle to maintain a distance between the vehicle and theat least one neighboring vehicle to be at least a predetermined minimumdistance to avoid colliding with the at least one neighboring vehicle.FIGS. 6A-6C show exemplary obstacles that may be encountered byvehicles. Once the obstacles are identified, a warning can be displayedon the advertisement display with a safety message for other vehicles orpedestrians as a public service. Similarly, for ridesharing companies, amessage can be placed on the ad display board to match the vehicle withthe ridesharing customer to ensure the customer enters the rightvehicle.

FIGS. 7A-7H illustrate an exemplary process to fuse data for 3D modelsused for car navigation. FIG. 7A shows an exemplary system that performsdata fusion based on sensor based detection of objects, change inweather and traffic, and holiday/emergency conditions, among others. Theprocess checks all the sensors for change in weather (2004), detectionof object (2002) and the GPS for current traffic conditions (2006). Foreach given sensor for detecting objects in a vehicle's environment, theprocess generates a 3D model of the given sensor's field of view;obstacle information from front cars using vehicle-vehicle communication(DRSC); neighboring car driver preference information; trafficinformation including emergency information. The process can adjust oneor more characteristics of the plurality of 3D models based on thereceived weather information to account for an impact of the actual orexpected weather conditions on one or more of the plurality of sensors.After the adjusting, aggregating, by a processor, the plurality of 3Dmodels to generate a comprehensive 3D model; combining the comprehensive3D model with detailed map information; and using the combinedcomprehensive 3D model with detailed map information to maneuver thevehicle. In FIG. 7A, the process checks sensors for object detection(2008) and then checks for confirmations from other vehicles over V2Vcommunication such as DSRC and then generates 3D model therefrom. Theprocess can also check for weather change (2004) and correlate theweather change to generate an updated 3D model. Similarly, the processintegrates traffic flow information (2006) and updates the 3D model asneeded. FIG. 7B shows an exemplary process for identifying the object,while FIG. 7C-7H show in more details the object modeling process. Theprocess checks sensors for object detection and scans the object against3D library for matches. If a match is found, the process sets the objectto the object in the library, and otherwise the process performs abest-guess of what the object is and send the object identification forsubsequent 3D modeling use.

FIGS. 8A-8F show exemplary detection of objects outside of the vehicleand guidance on their handling. The detected objects can includeautomobile, a pedestrian, structure, or a bicycle, for example. Thesystem assists the driver by identifying the objects as potential“threats” and recommend options for the driver. For example, the systemcan perform the following:

-   -   detecting an object external to a vehicle using one or more        sensors;    -   determining a classification and a state of the detected object;    -   estimating the destination of the object;    -   predicting a likely behavior of the detected object based on        prior behavior data and destination;    -   preparing the vehicle to respond based at least in part on the        likely behavior of the detected object; and    -   flashing a warning to the driver of the other warning using the        ad display.

FIG. 8A shows an exemplary process to identify a vehicle based on the 3Dmodels created in FIGS. 7A-7H. FIG. 8B shows an exemplary handling wherethe detected object is an automobile—the classification of the detectedobject includes the type of automobile. FIG. 8C shows a process toretrieve prior behavior data of the detected object by identifying atleast one of a logo, a bumper sticker, or a license plate. Suchinformation is then used to look up vehicle behavior. Public informationsuch as driving ticket and Insurance information can be extracted to seeif the driver has a bad driving history and if so the system can take adefensive driving posture. FIG. 8D shows an exemplary process todetermine the state of the object. For example, the state of thedetected object can be related to at least one of: location, trafficlane in which the detected object is traveling, speed, acceleration,entry onto a road, exit off of a road, activation of headlights,activation of taillights, or activation of blinkers. The behavior datais based on movement data for a plurality of other objects at one ormore locations. The movement data are tracked using one of: satelliteimagery, roadside cameras, on-board GPS data, or sensor data acquiredfor other nearby vehicles. FIG. 8E shows an exemplary process toidentify predict other driver/rider behavior, while FIG. 8F generatesproposed response to the object's expected behavior. The system can senda driver recommendation or vehicle command to orient the vehicleincludes positioning the vehicle at a predetermined distance from thedetected object, the predetermined distance being based, at least inpart, on the classification of the detected object. The likely behaviorof the detected object can be provided as a probability of the detectedobject entering to one or more states. The process includes receivingupdated behavior data; and wherein predicting the likely behavior of thedetected object is based at least in part on the updated behavior data.The driver can be informed of the options using haptic interface or aheads-up display. The process can also share the likely behavior of theobject to neighboring vehicles using vehicle-to-vehicle communication.

The process may cause the vehicle to take particular actions in responseto the predicted actions of the surrounding objects. For example, ifother car is turning at the next intersection, the process may slow thevehicle down as it approaches the intersection. In this regard, thepredicted behavior of other objects is based not only on the type ofobject and its current trajectory, but also based on some likelihoodthat the object may obey traffic rules or pre-determined behaviors. Inanother example, the process may include a library of rules about whatobjects will do in various situations. For example, a car in a left-mostlane that has a left-turn arrow mounted on the light will very likelyturn left when the arrow turns green. The library may be built manually,or by the vehicle's observation of other vehicles (autonomous or not) onthe roadway. The library may begin as a human built set of rules whichmay be improved by the vehicle's observations. Similarly, the librarymay begin as rules learned from vehicle observation and have humansexamine the rules and improve them manually. This observation andlearning may be accomplished by, for example, tools and techniques ofmachine learning. In addition to processing data provided by the varioussensors, the computer may rely on environmental data that was obtainedat a previous point in time and is expected to persist regardless of thevehicle's presence in the environment. For example, the system can usehighly detailed maps identifying the shape and elevation of roadways,lane lines, intersections, crosswalks, speed limits, traffic signals,buildings, signs, real time traffic information, or other such objectsand information. For example, the map information may include explicitspeed limit information associated with various roadway segments. Thespeed limit data may be entered manually or scanned from previouslytaken images of a speed limit sign using, for example, optical-characterrecognition. The map information may include three-dimensional terrainmaps incorporating one or more of objects listed above. For example, thevehicle may determine that another car is expected to turn based onreal-time data (e.g., using its sensors to determine the current GPSposition of another car) and other data (e.g., comparing the GPSposition with previously-stored lane-specific map data to determinewhether the other car is within a turn lane). These objects may haveparticular behavior patterns that depend on the nature of the object.For example, a bicycle is likely to react differently than a motorcyclein a number of ways. Specifically, a bicycle is more likely to makeerratic movements when compared with a motorcycle, but is much slowerand thus can be handled with ease compared to a speeding motorcycle. Foreach classification, the object data may also contain behaviorinformation that indicates how an object having a particularclassification is likely to behave in a given situation. Vehicle maythen autonomously respond to the object based, in part, on the predictedbehavior.

FIG. 9A shows an exemplary system for crowd-sourcing navigation data.The system includes a crowdsourcing server in communication with aplurality of vehicles 1 . . . n. The vehicles in FIG. 9A performspeer-to-peer discovery and crowd-sourced navigation as shown in FIG. 9B.The system receives proximity services for a group of vehicles travelinga predetermined route using peer-to-peer discovery, receivescrowdsourcing data from said plurality of vehicles, sharingcrowdsourcing data to the group of vehicles (or a subsequent group ofvehicles) traveling the route of interest. Such information can be usedin providing navigation guidance to the vehicle traveling the routeusing the crowdsourced data.

Crowd-Sourced Map Updating and Obstacle Annotating and Display PublicService Warnings on the Ad Display

Next, a system to crowd-source the updates of precision maps with datafrom smart vehicles is detailed. In embodiments, crowd-sourced obstacledata can be used to update a map with precision. The obstacles can berocks, boulders, pot-holes, manhole, utility hole, cable chamber,maintenance hole, inspection chamber, access chamber, sewer hole,confined space or can be water pool or rising tidal waves that affectthe road as detected by a plurality of vehicles. Such crowd-sourcedinformation is updated into the map and annotated by time, weather andperiodicity. The detected obstacle information may include a geographiclocation of the vehicle and a predetermined map of the road. Thecomputer system may determine the geographic location of the obstacleby, for example, using a laser rangefinder or light detection andranging (LIDAR) unit to estimate a distance from the obstacle to the atleast two objects near the vehicle and determining the geographiclocation of the obstacle using triangulation, for example. Suchinformation is updated into the map system and marked as temporal.During use, if recent vehicles take defensive driving around thetemporary obstacle, the map adds the obstacles to the map for the routeguidance module to advise vehicles. If recent vehicles drive the road asthough the obstacle does not exist, the system removes the obstacle fromthe map database, but keeps track of the history in case it is aperiodic obstacle. The obstacle information is also reported togovernment agency for repair/maintenance.

In another embodiment, if vehicles drive through the lane with a smoothline or curve, but abruptly brakes, the system infers that the road hasdefects or potholes, for example, and the bad infrastructure is reportedfor path planning (to add more travel time, or to change the route toavoid the bad road infrastructure if it is long.

The new information is used to update a digital map that lacks thecurrent information or that contains inaccuracies or may be incomplete.The digital map stored in the map database may be updated using theinformation processed by a map matching module, matched segment module,and unmatched segment module. The map matching module, once it hasreceived obstacle location and GPS traces, processes obstacle locationsand GPS traces by matching them to a road defined in the digital map.The map matching module matches the obstacles and the GPS traces withthe most likely road positions corresponding to a viable route throughthe digital map by using the processor to execute a matching algorithm.In one example, the matching algorithm may be a Viterbi matchingalgorithm. Where the GPS traces do match a road defined in the digitalmap, the matched trace to which the GPS traces match and obstacleinformation are sent to the matched segment module for furtherprocessing as will be described below. Where the GPS traces do not matcha road defined in the digital map, the unmatched trace to which the GPStraces are correlated with and the obstacle position information aresent to the unmatched segment module for further processing. The matchedsegment module and unmatched segment module both provide metadata to themap updating module. The metadata may include obstacle metadata roadgeometry refinement metadata, road closure and reopening metadata,missing intersection metadata, missing road data and one-way correctionmetadata. The map updating module updates the digital map in the mapdatabase.

The process to update maps using crowd-sourced data may begin with theunmatched segment module clustering the unmatched GPS traces receivedfrom the map matching module. Many available algorithms may be suitablefor this process, but in one example, an agglomerative clusteringalgorithm that iteratively compares GPS traces with each other andcombines those that fall within a pre-determined tolerance into acluster may be used. One example of such and algorithm uses theHausdorff distance as its distance measure in the clustering algorithm.Once the cluster is selected, the unmatched segment module may produce asingle road geometry for a cluster of unmatched GPS traces using acenterline fitting procedure in which the single road geometry describesa new road segment with the obstacle which is not described in thecurrent map database. In one example, a polygonal principal curvealgorithm or a Trace Clustering Algorithm (TC1) algorithm can be used.The digital map can be modified to include the new road, includingpossibly new intersections in the base map and any associated pointersor indices updated.

Lane Marking Visibility Handling Using the Ad Display

In case of poor visibility (fog or snow) that can confuse driver andautomated driving system, the advertising boards can show road curvatureto cars that follow it. In some embodiments, a lead vehicle identifieslane information that may include lane markings on the road, and thecomputer system may use one or more sensors to sense the lane markings.At some point, the lead vehicle may determine that the lane informationhas become unavailable or unreliable. For example, severe fog may bepresent and severely affect the lane markings. In other examples, thevehicle may no longer be able to detect the lane markings on the road,the vehicle may detect contradictory lane markings on the road, thevehicle may no longer be able to determine a geographic location of thevehicle, and/or the vehicle may not be able to access a predeterminedmap of the road. Other examples are possible as well.

In response to determining that the lane information has becomeunavailable or unreliable, the computer system may use at least onesensor to monitor at least one neighboring vehicle, such as aneighboring vehicle in a neighboring lane or a neighboring vehiclebehind the vehicle that is part of the flock. The computer system maythen control the vehicle to maintain a distance between the vehicle andthe at least one neighboring vehicle to be at least a predeterminedminimum distance and even if the vehicle is unable to rely on the laneinformation to estimate a location of the lane on the road, the vehiclemay avoid colliding with the at least one neighboring vehicle.

In other embodiments, the lane information may include a geographiclocation of the vehicle and a predetermined map of the road. Thecomputer system may determine the geographic location of the vehicle by,for example, querying a location server for the geographic location ofthe vehicle. Alternatively, if the predetermined map indicates ageographic location of at least two objects near the vehicle, thecomputer system may determine the geographic location of the vehicle by,for example, using a laser rangefinder or light detection and ranging(LIDAR) unit to estimate a distance from the vehicle to the at least twoobjects near the vehicle and determining the geographic location of thevehicle using triangulation. Other examples are possible as well. In anycase, the computer system may then locate the geographic location of thevehicle on the predetermined map to determine a location of the lanerelative to the geographic location of the vehicle.

In still other embodiments, the lane information may be derived from aleading vehicle that is in front of the vehicle in the lane andcorrelation with other information such as map data and independent laneanalysis to prevent the blind-following-the blind situation. Thecomputer system may estimate a path of the leading vehicle using, forexample, a laser rangefinder and/or a LIDAR unit. Other examples arepossible as well. Once the computer system has estimated the path of theleading vehicle, the computer system may estimate the location of thelane based on the estimated path. For example, the computer system mayestimate the location of the lane to include the estimated path (e.g.,extend by half of a predetermined lane width on either side of theestimated path). Other examples are possible as well.

In some embodiments, the computer system may maintain a predeterminedthreshold for the lane information, and the computer system maydetermine that the lane information has become unavailable or unreliablewhen the computer system detects that a confidence of the laneinformation (e.g., how confident the computer system is that the laneinformation is reliable) is below the predetermined threshold. In someembodiments, the computer system may additionally maintain apredetermined time period for the lane information, and the computersystem may determine that the lane information has become unavailable orunreliable when the computer system detects that a confidence of thelane information is below the predetermined threshold for at least thepredetermined amount of time.

Upon determining that the lane information has become unavailable orunreliable, the computer system may use at least one sensor to monitorat least one neighboring vehicle. The at least one neighboring vehiclemay include, for example, a neighboring vehicle in a lane adjacent tothe lane in which the vehicle is traveling. As another example, the atleast one neighboring vehicle may include a neighboring vehicle behindthe vehicle in the lane in which the vehicle is traveling. As stillanother example, the at least one neighboring vehicle may include afirst neighboring vehicle and a second neighboring vehicle, each ofwhich may be either in a lane adjacent to the lane in which the vehicleis traveling or behind the vehicle in the lane in which the vehicle istraveling. Other examples are possible as well.

When the lane information has become unavailable or unreliable, thecomputer system may control the vehicle to maintain a distance betweenthe vehicle and the at least one neighboring vehicle to be at least apredetermined distance. The predetermined distance may be, for example,a distance determined to be a safe distance and/or a distanceapproximately equal to the difference between a predetermined lane widthand a width of the vehicle. Other predetermined distances are possibleas well.

In order to maintain the distance between the vehicle and the at leastone neighboring vehicle to be at least the predetermined distance, thecomputer system may continuously or periodically use the at least onesensor on the vehicle to monitor the distance between the vehicle andthe at least one neighboring vehicle. The computer system may monitorthe distance between the vehicle and the at least one neighboringvehicle using, for example, a laser rangefinder and/or LIDAR unit. Ifthe distance between the vehicle and the at least one neighboringvehicle becomes less than the predetermined distance, the computersystem may move the vehicle away from the at least one neighboringvehicle in order to maintain the distance between the vehicle and the atleast one neighboring vehicle to be at least the predetermined distance.

In some embodiments, in addition to maintaining the distance between thevehicle and the at least one neighboring vehicle to be at least thepredetermined distance, the computer system may additionally maintainthe distance between the vehicle and the at least one neighboringvehicle to be within a predetermined range of the predetermineddistance. In these embodiments, if the distance between the vehicle andthe at least one neighboring vehicle becomes too large (e.g., no longerwithin the predetermined range of the predetermined distance), thecomputer system may move the vehicle closer to the at least oneneighboring vehicle. This may, for example, prevent the vehicle fromdrifting so far away from the neighboring vehicle that the vehicledrifts into a lane on the opposite side of the vehicle from theneighboring vehicle.

As noted above, in some embodiments the at least one vehicle may includea first neighboring vehicle and a second neighboring vehicle. In theseembodiments, maintaining the distance between the vehicle and the atleast one neighboring vehicle may involve maximizing both a firstdistance between the vehicle and the first neighboring vehicle and asecond distance between the vehicle and the second neighboring vehicle(e.g., such that the vehicle remains approximately in the middle betweenthe first neighboring vehicle and the second neighboring vehicle). Eachof the first distance and the second distance may be at least thepredetermined distance.

In some embodiments, in addition to maintaining the distance between thevehicle and the at least one neighboring vehicle to be at least thepredetermined distance, the computer system may determine an updatedestimated location of the lane. To this end, the computer system may usethe at least one sensor to monitor at least a first distance to the atleast one neighboring vehicle and a second distance to the at least onevehicle. Based on the first distance and the second distance, thecomputer system may determine a first relative position and a secondrelative position (e.g., relative to the vehicle) of the at least oneneighboring vehicle. Based on the first relative position and the secondrelative position, the computer system may estimate a path for the atleast one neighboring vehicle. The computer system may then use theestimated path to determine an updated estimated location of the lane.For example, in embodiments where the at least one neighboring vehicleis traveling in a lane adjacent to the lane in which the vehicle istraveling, the computer system may determine the estimated location ofthe lane to be substantially parallel to the estimated path (e.g., thelane may be centered on a path that is shifted from the estimated pathby, e.g., a predetermined lane width and may extend by half of thepredetermined lane width on either side of the path). As anotherexample, in embodiments where the at least one neighboring vehicle istraveling behind the vehicle in the lane in which the vehicle istraveling, the computer system may determine the estimated location ofthe lane to be an extrapolation (e.g., with constant curvature) of theestimated path. Other examples are possible as well.

In some embodiments, the computer system may additionally use a speedsensor to monitor a speed of the at least one neighboring vehicle andmay modify a speed of the vehicle to be less than the speed of the atleast one neighboring vehicle. This may allow the vehicle to be passedby the at least one neighboring vehicle. Once the at least oneneighboring vehicle has passed the vehicle, the at least one neighboringvehicle may become a leading vehicle, either in a lane adjacent to thelane in which the vehicle is traveling or a leading vehicle that is infront of the vehicle in the lane in which the vehicle is traveling, andthe computer system may estimate the location of the lane of the roadbased on an estimated path of the leading vehicle, as described above.

In some embodiments, the computer system may begin to monitor the atleast one neighboring vehicle only in response to determining that thelane information has become unavailable or unreliable. In theseembodiments, prior to determining that the lane information has becomeunavailable or unreliable, the computer system may rely solely on thelane information to estimate the location of the lane. In otherembodiments, however, the computer system may also monitor the at leastone neighboring vehicle prior to determining that the lane informationhas become unavailable or unreliable. In these embodiments, the computersystem may additionally use the distance to the at least one neighboringvehicle to estimate the location of the lane in which the vehicle istraveling. For example, if the at least one neighboring vehicle istraveling in a lane adjacent to the lane in which the vehicle istraveling, the computer system may determine that the lane does notextend to the at least one neighboring vehicle. As another example, ifthe at least one neighboring vehicle is traveling behind the vehicle inthe lane in which the vehicle is traveling, the computer system maydetermine that the lane includes the at least one neighboring vehicle.Other examples are possible as well. Alternatively, in theseembodiments, prior to determining that the lane information has becomeunavailable or unreliable, the computer system may simply use thedistance to the at least one neighboring vehicle to avoid collisionswith the at least one neighboring vehicle.

Further, in some embodiments, once the vehicle begins to monitor the atleast one neighboring vehicle, the computer system may stop using thelane information to estimate the location of the lane in which thevehicle is traveling. In these embodiments, the computer system may relysolely on the distance to the at least one neighboring vehicle to avoidcollisions with the at least one neighboring vehicle until the laneinformation becomes available or reliable. For example, the computersystem may periodically attempt to obtain updated lane information. Oncethe computer system determines that the lane information has becomeavailable or reliable, the lane information has become available orreliable, the computer system may once again rely on the updatedestimated location of the lane and less (or not at all) on the distanceto the at least one neighboring vehicle. The computer system maydetermine that the updated lane information is reliable when, forexample, the computer system determines that a confidence of the updatedlane information is greater than a predetermined threshold. Thepredetermined threshold may be the same as or different than thepredetermined threshold.

FIG. 10 shows a sequence of cars displaying ads that are sequenced likea movie. This type of ad has a number of vehicles traveling into apredetermined zone, with each vehicle ad display showing a differentimage/video as part of a coordinated sequence. The group of cars canfolllow flock control behavior to display sequenced ads as a collective,and more on the flock operation is detailed in commonly owned U.S. Pat.No. 9,711,050, the content of which is incorporated by reference.

Reach and frequency buying has a sequencing tool that allows the adbuyer to arrange up to 50 ads in a certain order to be shown to anaudience. People in a target audience will have the opportunity to seeads in the order the advertiser sets. For example, ad #1 will bedelivered to them before ad #2 is delivered. The system can controlwhether or not a person has to have engaged with the content being shownorder to see the next ads in your sequence. Sequence the ads of FIG. 10can be used when the advertisers want to tell a story or presentinformation to people in a specific order. For example: If theadvertiser want to build a narrative with ads, the system can show onead that tells the beginning of the story first and then show additionalads afterward that continue to tell the story in a set order. Theaudience will see ad #1 before they see ad #2, and so on. In oneembodiment, for sequencing the reach and frequency ads:

Go to Ads Manager and create your ad campaign.

Reserve a reach and frequency campaign.

From the Ads Manager home screen, select the ad just booked and thenchoose which ad set to be sequence.

After selecting the ad set, select the Edit icon.

In the editing panel, go to the Delivery section and choose Sequencedand then Add for each ad to include in ad sequence.

The number of ads included depends on a campaign schedule and howfrequently the advertiser chooses to show the ads. For example, if acampaign is three weeks long and the advertisers have set up reach andfrequency balance for showing an ad once every seven days, theadvertisers have three opportunities (or one opportunity per week) toshow a new ad before the campaign ends.

FIG. 11 illustrate a typical network environment 4100 in which thesystems, methods, and computer program products may be implemented,according to embodiments as disclosed herein. In an embodiment, theenvironment 4100 includes a plurality of vehicles 4102. The vehicle 4102described herein can be configured to include a advertising monitoringunit 4104 installed thereon. The monitoring device may be selfcontained, such as a single unit mounted on a windshield or dashboard ofthe vehicle 4102. Alternatively, the monitoring device 4104 may includemultiple components, such as a processor or central unit mounted under acar seat or in a trunk of the vehicle and a user interface mounted on adashboard or windshield. Similarly, the monitoring unit 4104 may have aself-contained antenna in the unit or may be connected to remotelymounted antennas for communication with remote systems.

Further, the advertising monitoring units 4104 may be connected to anon-board diagnostic system or data bus in the vehicle 4104. Informationand behavior data associated with the driver may be collected from theon-board diagnostic system. The advertising monitoring system mayreceive inputs from internal and external sources and sensors such asaccelerometers, global positioning systems (GPS), vehicle on-boarddiagnostic systems, seatbelt sensors, wireless device, or cell phone usedetectors, alcohol vapor detectors, or trans-dermal ethanol detection.Further, the details related to the advertising monitoring unit 4104 aredescribed in conjunction with the FIG. 12.

Further, the information may be exchanged between advertising monitoringunit 104 and central monitoring system or server 4106 in real-time or atintervals. For example, the vehicle behavior parameters may betransmitted to server 4106 via a communication network 4108. In anembodiment, the communication network 4108 described herein can includefor example, but not limited to, a cellular, satellite, Wi-Fi,Bluetooth, infrared, ultrasound, short wave, microwave, global systemfor mobile communication, or any other suitable network. The informationsent to the server 4104 may then be forwarded with one or more insuranceproviders 4110. The server 4106 can be configured to process the vehiclebehavior parameters and/or store the data to a local or remote database.The drivers or insurance provider can access the data on the server4106. In some embodiments, the data captured by monitoring unit 4104 inthe vehicle 4102 may be transmitted via a hardwired communicationconnection, such as an Ethernet connection that is attached to vehicle4102 when the vehicle is within a service yard or at a base station ornear the server 4106. Alternatively, the data may be transferred via aflash memory, diskette, or other memory device that can be directlyconnected to the server 4106.

In one embodiment of the invention, the data captured by advertisingmonitoring unit 4104 can be used to monitor, provide feedback, mentor,provide recommendations, adjust insurance rates, and to analyze adriver's behavior during certain events. For example, if vehicle 4102 isoperated improperly, such as speeding, taking turns too fast, collidingwith another vehicle, or driving in an unapproved area, then theadvertising monitoring unit 4104 or server 4106 may adjust the insurancerates for the driver and provide feedback and suggestions to the driver,such as to improve the diving skills. Additionally, if the driver'sbehavior is inappropriate or illegal, such as not wearing a seatbelt orusing a cell phone while driving then feedback and suggestions can beprovided to the driver to improve the diving skills.

In an embodiment, the insurance price may be adjusted based on thevehicle behavior. For example, if an insurance company, supervisor, orother authority determines that the driver is uninsured, underinsured,lacking coverage required in a particular jurisdiction, that thedriver's insurance premiums are delinquent, and/or if the vehicle is notproperly registered and/or delinquent in registration with the state,then the advertising monitoring unit 102 may be directed to disable ordeactivate the vehicle. Alternatively, the advertising monitoring unit102 can provide feedback and recommendations to the driver if it isdetermined that the vehicle behavior is uninsured, underinsured, lackingcoverage required in a particular jurisdiction, or that the driver'sinsurance premiums are delinquent. In an embodiment, the driver'sbehavior is typically evaluated while driving the vehicle 102 with theadvertising monitoring unit 104 installed thereon. After receiving thevehicle behavior data from the advertising monitoring unit 104, theinsurance rates can be adjusted accordingly.

FIG. 12 is a diagram illustrating generally, a portion of vehicle 4200alone with possible locations of sensors, cameras, and/or othertechnologies, according to embodiments described herein. In anembodiment, exemplary mounted locations for the advertising monitoringunit 4104 are illustrated, such as on a dashboard 4202, windshield 4204,headliner 4206, surface 4208, corner 4210. It will be understood thatall or parts of the advertising monitoring unit 4104 can be mounted inany other location that allows for audio and/or visual feedback to thedriver of the vehicle 4102 while the vehicle is in operation. Theadvertising monitoring unit 4104 is illustrated as being coupled toon-board diagnosis, from which it may receive inputs associated with thedriver and vehicle operating parameters. The advertising monitoringunits such as 4202, 4204, 4206, 4208, and 4210 can be coupled toon-board diagnosis (not shown). Moreover, the advertising monitoringsystem may be coupled to other sensors, such as a sensor for detectingthe operation and use of a cellular or wireless device in the vehicle4102.

In an embodiment, the advertising monitoring units can be configured toinclude for example, but not limited to, accelerometer, cameras,gyroscope, magnetometer, and the like sensors. In an embodiment, theaccelerometer can include at least one accelerometer for measuring alateral (sideways), longitudinal (forward and aft) and verticalacceleration in order to determine whether the driver is operating thevehicle in an unsafe or aggressive manner. For example, excessivelateral acceleration may be an indication that the driver is operatingthe vehicle at an excessive speed around a turn along a roadway.Furthermore, it is possible that the driver may be traveling at a speedwell within the posted speed limit for that area of roadway. However,excessive lateral acceleration, defined herein as “hard turns,” may beindicative of aggressive driving behavior by the driver and maycontribute to excessive wear on tires and steering components as well aspotentially causing the load such as a trailer to shift and potentiallyoverturn.

As such, it can be seen that monitoring such vehicle behavior byproviding feedback and recommendations to the vehicle to increase adrevenue. For example, driving too fast may reduce views and thus reducerevenue. Driving in high traffic areas may increase revenue, whiledriving in scenic areas may increase revenue as well. In addition, thesystem can recommend paths to reduce wear and tear on the vehicle andultimately reduce fleet maintenance costs as well as reduce insurancecosts and identify at risk drivers and driving behavior to fleetmanagers.

In one aspect, the advertising monitoring system may be in datacommunication with an on board diagnostic (OBD) system of the vehiclesuch as via a port. In some vehicle models, the advertising monitoringsystem is in data communication with a controller area network (CAN)system (bus) to allow acquisition of certain driver and vehicleoperating parameters including, but not limited to, vehicle speed suchas via the speedometer, engine speed or throttle position such as viathe tachometer, mileage such as via the odometer reading, seat beltstatus, condition of various vehicle systems including anti-lock-braking(ABS), turn signal, headlight, cruise control activation and a multitudeof various other diagnostic parameters such as engine temperature, brakewear, and the like. The OBD or CAN allows for acquisition of theabove-mentioned vehicle parameters for processing thereby and/or forsubsequent transmission to the server 4106.

In an embodiment, the advertising monitoring system may also include aGPS receiver (or other similar technology designed to track location)configured to track the location and directional movement of the driverin either real-time or over-time modes. As is well known in the art, GPSsignals may be used to calculate the latitude and longitude of a driveras well as allowing for tracking of driver movement by inferring speedand direction from positional changes. Signals from GPS satellites alsoallow for calculating the elevation and, hence, vertical movement, ofthe driver.

In an embodiment, the advertising monitoring unit may further include amobile data terminal (MDT) mounted for observation and manipulation bythe driver, such as near the vehicle dash. The MDT can be configured toinclude an operator interface such as a keypad, keyboard, touch screen,display screen, or any suitable user input device and may furtherinclude audio input capability such as a microphone to allow voicecommunications. The advertising monitoring unit receives inputs from anumber of internal and external sources. The OBD/CAN bus, which providesdata from the vehicle's on-board diagnostic system, including engineperformance data and system status information. A GPS receiver provideslocation information. The CDR, XLM, or accelerometers provideinformation regarding the vehicle's movement and driving conditions. Anynumber of other sensors, such as but not limited to, a seat belt sensor,proximity sensor, advertising monitoring sensors, or cellular phone usesensors, also provide inputs to the advertising monitoring system.

In an embodiment, the advertising monitoring system may have any type ofuser interface, such as a screen capable of displaying messages to thevehicle's driver or passengers, and a keyboard, buttons or switches thatallow for user input. The system or the user interface may have one ormore status LEDs or other indicators to provide information regardingthe status of the device's operation, power, communications, GPS lock,and the like. Additionally, the LEDs or other indicators may providefeedback to the driver when a driving violation occurs. Additionally,monitoring system may have a speaker and microphone integral to thedevice.

In an embodiment, the monitoring system may be self-powered, such as bya battery, or powered by the vehicle's battery and/or power generatingcircuitry. Access to the vehicle's battery power may be by accessing thepower available on the vehicle's OBD and/or CAN bus. The advertisingmonitoring system may be self-orienting, which allows it to be mountedin any position, angle or orientation in the vehicle or on thedashboard. In an embodiment, the advertising monitoring systemdetermines a direction of gravity and a direction of driver movement anddetermines its orientation within the vehicle using this information. Inorder to provide more accurate measurements of vehicle behavior, thepresent invention filters gravitational effects out of the longitudinal,lateral and vertical acceleration measurements when the vehicle is on anincline or changes its horizontal surface orientation. Vehicle behaviorcan be monitored using the accelerometer, which preferably will be atri-axial accelerometer. Acceleration is measured in at least one oflateral, longitudinal and/or vertical directions over a predeterminedtime period, which may be a period of seconds or minutes. Anacceleration input signal is generated when a measured accelerationexceeds a predetermined threshold.

It will be understood that the present invention may be used for bothfleets of vehicles and for individual drivers. For example, theadvertising monitoring system described herein may be used by insuranceproviders to monitor, recommend, provide feedback, and adjust insurancerates based on the driving. A private vehicle owner may also use thepresent invention to monitor the vehicle behavior and user of thevehicle. For example, a parent may use the system described herein tomonitor a new driver or a teenage vehicle behavior.

An embodiment of the invention provides real-time recommendations,training, or other feedback to a driver while operating the vehicle. Therecommendations are based upon observed operation of the vehicle and areintended to change and improve vehicle behavior by identifying improperor illegal operation of the vehicle. The advertising monitoring systemmay identify aggressive driving violations. For example, based upon theinputs from an acceleration or CDR, aggressive driving behavior can bedetected, such as exceeding acceleration thresholds in a lateral,longitudinal, or vertical direction, hard turns, hard acceleration orjackrabbit starts, hard braking, and/or hard vertical movement of thevehicle.

Further, in an embodiment, the sensor and camera described herein can beconfigured to communicate with the vehicle entertainment system.Typically, this functionality includes pre-installed software or auser-downloadable application from a network source (such as Apple'siTunes or Google's Android Market). The system functionality may includemapping functions, directions, landmark location, voice-control, andmany other desirable features. When such mobile computing device isplaced within the vehicle then a convenient vehicle entertainment systemassociated with the vehicle can be provided. In an embodiment, a remoteswitch can be used to initiate the vehicle entertainment softwareapplication by communicating with the cameras/sensors located in thevehicle and/or software residing on the mobile computing device. Remoteswitch described herein can include one of a number of well-known remoteswitches that uses wireless or wired technology to communicate withmobile computing device. For example, remote switch may include forexample, but not limited to, a Bluetooth, RF, infrared, or otherwell-known wireless communication technology, or it may be connected viaone or more wires to mobile computing device. The switch may be locatedon any vehicle interior surface, such as on a steering wheel, visor,dashboard, or any other convenient location.

FIG. 13 is a diagram 4300 illustrating generally, possible locations ofsensors, cameras, and/or other technologies, according to embodimentsdescribed herein. FIG. 14 is a sequence diagram illustrates generally,operations 300 performed by the system as described in FIG. 11,according to embodiments described herein. In an embodiment, at 4402,the advertising monitoring unit 104 can be configured to monitor thebehavior of the driver. The system can be configured to include theadvertising monitoring unit 4104 installed in the vehicle 102 to monitorthe behavior parameters of the driver while the vehicle 4102 is beingdriven. The vehicle 4102 can include cameras, gyroscope, magnetometer,accelerometer, and other sensors installed thereon to monitor thebehavior parameter of the driver. In an embodiment, the cameras orsensors may be placed at any place in the vehicle, such as for exampleat four corners of the front windshield, in a way that it can directlycapture the behavior parameters of the driver. For example, based on thedriver gestures, the cameras can detect finger position to detect thatdriver is pointing at a particular object or vehicle and searches theinternet for the vehicle. Further, in an embodiment, a flexible displayfilm adhesively secured on the front windshield. The display can be usedcontrolled by a computer to display info in a discrete way that may nottake driver's eyes off the road and opposing vehicles. In an embodiment,at 4404, the advertising monitoring unit 4102 can be configured totransmit the behavior parameters of the driver to the server 4106. In anembodiment, the vehicle behavior parameters described herein can includefor example, but not limited to, vehicle speed, vehicle accelerations,driver location, seatbelt use, wireless device use, turn signal use,driver aggression, detection of CO2 vapor, detection of alcohol, driverseating position, time, and the like. In an embodiment, at 4406, theserver 4106 can be configured to transmit the vehicle behaviorparameters to one or more insurance providers. In an embodiment, at4408, the server 4106 can be configured to analyze the vehicle behaviorparameters and adjust the insurance rates for the driver. For example,if the driver is driving roughly by drinking alcohol then the insurancerate may get decreased. In an embodiment, at 4410, the server 4106 canbe configured to match the vehicle behavior preferences with similar orsubstantially similar preferences of other drivers. The server 4104 canbe configured to generate action recommendations best matching thebehavior of the driver. In an embodiment at 4412, the server 4106 can beconfigured to provide the generated recommendations to the driver. Basedon the vehicle behavior parameters the sever 4106 provides feedback andrecommendations to the driver, such as to improve the driving skills.Further, in an embodiment, a flexible display film adhesively secured onthe front windshield. The display can be used controlled by a computerto display info in a discrete way that may not take driver's eyes offthe road and opposing vehicles. In an embodiment, at 4414, the server4106 can be configured to frequently monitor the behavior parametersassociated with the driver. Any changes in the behavior parameters canaffect the overall system performance and the driver experience. Theserver 4106 can be configured to frequently monitor and dynamicallyupdate the insurance rate and action recommendations, which in turnhelps the driver for effectively improving the driving skills.

FIG. 14 shows an embodiment for providing insurance to ride-sharing carsor motorbikes based on the driver behavior. In one embodiment, thesystem rents/leases motorbikes with the advertisement boards on the backor the side. Thus, in addition to ad revenue, the system enables lastmile delivery of items (food/shipped goods . . . ) using a crowd-sourceddrivers who can use the motorbikes/cars to earn a living throughridesharing as well as advertising revenue sharing. The system cancharge the lease rate based on driver prior history, and can dynamicallychange the insurance on the rented vehicle based on most recentdriving/riding behavior. The system of FIG. 14 can increase/decreaseinsurance charged to the lease based on driver aggressiveness, forexample.

FIG. 15 is a diagram 4500 illustrates generally, an overview of areasonable action determination system that may allow drivers to obtainaction recommendations based on the vehicle behavior parameters,according to embodiments disclosed herein. In an embodiment, the vehiclebehavior parameters can be used to provide customized recommendations todrivers by comparing the vehicle behavior parameters with other driverswho has similar or substantially similar behavior parameters. Unlikeconventional system, the server 106 can be configured to adaptivelygenerate action recommendations for the driver based on the behaviorparameters. The server 106 can be configured to match the behaviorparameters of the drivers to similar behavior parameters of the one ormore drivers, such as to provide personalized action recommendations tothe driver. In an embodiment, the recommendations can be filtered inadvance of display. In an embodiment, filtered recommendations may bederived from the sources such as for example, but not limited to, thosesources that have added the data within a specified time, from thosesources that share specific similarities with the sources, those sourcesthat have been preselected by the driver as relevant, those sources thatare selected as friends or friends of friends, and the like, thosesources that are determined to provide valuable reviews/ratings or arespecifically declared to be experts within the system or by the driver,or those users that have entered at least a minimum amount of data intothe system.

FIG. 16 is a diagram 4600 illustrates generally, an overview ofpreferences matching by the server 4106, according to embodimentsdisclosed herein. FIG. 16 outlines reasonableness determinationfunctionality in accordance with an embodiment of the present invention.The system 4100 can monitor the vehicle behavior and uses the behaviordata to match with the behavior data of other sources and providereasonable recommendations to the driver. In an embodiment, thereasonableness recommendation rules may be established in therecommendation system such as described in the FIG. 16. Such rulesderived from, for example, but not limited to, automatic generationmachine learning, automatic generation using a generic algorithm,automatic generation using a neutral network, automatic generation usinga rule inference system, data mining, generation using a preset list ofrecommendations, and/or a vehicle behavior. In an embodiment, the sever106 can be configured to receive the recommendation rules such asunidirectional rules, bidirectional rules, generalized rules includingmulti-way rules, rules among items, rules among sets, rules amongcollections, rules with weight factors, rules with priorities,un-weighted and un-prioritized rules, and the like.

FIG. 17 is a flow chart illustrates generally, a method 4700 forselectively providing reasonable driving information to a serviceprovider, according to embodiments as disclosed herein. At step 4702,the autonomous behavior is monitored. The behavior data can includeexternal parameters and/or internal parameters. In an embodiment, theautonomous behavior data/parameters described herein can include forexample, but not limited to, vehicle speed, vehicle accelerations,driver location, seatbelt use, wireless device use, turn signal use,driver aggression, detection of ethanol vapor, driver seating position,time, and the like. In an embodiment, the behavior data can be over aperiod of hours, days, weeks, and so forth. In an embodiment, thebehavior data gathering can be continuous, at predefined intervals, orat random intervals. In accordance with some aspects, data can begathered while a vehicle is in operation and at other times (e.g., attwo a.m. to determine where the vehicle is parked overnight). In anembodiment, a change to an insurance premium and/or an insurancecoverage is prepared, at 4704. The change is based on one or more of thevehicle behavior data, wherein each item of vehicle behavior data canhave a different weight assigned. For example, data gathered related toweather conditions might be given less weight than data gathered relatedto user distractions (e.g., passengers, use of a mobile device whilevehicle is in operation, and so forth). In another example, excessivespeed might be assigned a higher weight than data related to safetyperformance of the vehicle. As such, data with a higher weight can begiven more consideration than data with a lower weight (e.g., dataassigned a higher weight can have a greater impact on the cost ofinsurance). Thus, if the user is traveling at (or below) the speed limitand speed is assigned a greater weight, then the safe speed will tend todecrease (or remain constant) the cost of insurance.

In an embodiment, the autonomous controller is notified of the change,at 4706. The notification can be in any perceivable format. In anexample, the notification is provided as a dashboard-mounted display. Inanother example, presenting the change can include displaying themodified cost of the insurance policy in a dashboard-mounted displayand/or a heads-up display. In an embodiment, a service provider isnotified of the change, at 708. At substantially the same time asnotifying the service provider (or trusted third party) of the change,parameters taken into consideration (and associated weight) can also beprovided. In such a manner, the service provider (or third party) canselectively further modify the cost of insurance, which can becommunicated to the user though the vehicle display or through othermeans.

The service provider (or third party) might be provided the changeinformation less often than the insurance cost change information isprovided to the user. For example, the user can be provided theinsurance cost change information dynamically and almost instantaneouslywith detection of one or more parameters that can influence theinsurance cost. However, the insurance provider (or third party) mightonly be notified of the change after a specified interval (or based onother intervals). For example, insurance cost changes might beaccumulated over a period of time (e.g., two weeks) and an average ofthe insurance cost changes might be supplied to insurance provider. Insuch a manner, the user has time to adjust parameters that tend toincrease (or decrease) the cost of insurance, which allows the user tohave more control over the cost of insurance.

In an embodiment, Vertical market specialization for insurance isprovided where markets are defined based on granular aspects of coverageand presented to one or more insurance subsystems to obtain quotes for acoverage premium. Such specialization allows insurance companies tocompete in more specific areas of insurance coverage, which allows formore accurate premium rates focused on the specific areas or one or morerelated scenarios. In addition, the granular aspects of coverage can beprovided to one or more advertising systems in exchange for furtherlowered rates, if desired.

According to an example, an insurance market can be defined based ongranular information received regarding an item, a related person, useof the item, etc. Based on the market, premium quotes can be obtainedfrom one or more insurance subsystems related to one or more insurancebrokers. In addition, rates can be decreased where the granularinformation can be provided to an advertising system, in one example. Inthis regard, targeted advertisements can additionally be presented tosystem related to requesting the insurance coverage. Policies can beautomatically selected based on preferences, manually selected using aninterface, and/or the like.

FIG. 18 is a diagram 4800 illustrates generally, an exemplary systemthat customizes insurance rates to correspond to behavior driver,according to embodiments as disclosed herein. In an embodiment, theserver 4106 can be configured to maintain a database component 4802including data related to different vehicle behaviors. Such leveragingfrom data banks enables insurance providers to bid in real time, andhence an owner and/or user of a vehicle can benefit from competitionamong various insurance providers, to obtain optimum rates. The serverincludes a rate adjustment component 4804 that in real time candetermine the various rates from a plurality of insurance providers 4110(1 to N, where N is an integer). In one particular aspect, a retrievalagent (not shown) associated with the rate adjustment component 4804 canpull insurance data from the insurance providers based on the contextualdata supplied thereto. For example, such contextual data can be datarecords related to vehicle behavior, the vehicle 4102 (such as auto shopservice records, current service status for the car, and the like), datarelated to the individual driver (such as health records, criminalrecords, shopping habits, and the like), data related to the environment(road condition, humidity, temperature, and the like) and data relatedto real time driving (frequency of braking, accelerating, intensity ofsuch actions, and the like).

The retrieval agent (not shown) can pull data from the insuranceproviders 4110 and further publish such data to enable a richinteraction between the users on a display or a within a writtencommunication environment. The retrieval agent can further generate aninstance for a connection with the insurance providers. Accordingly, aconnection instance can be employed by the rate adjustment component4804 to store connection information such as the state of dataconveyance, the data being conveyed, connection ID and the like. Suchinformation can additionally be employed to monitor progress of datatransfer to the written communication environment or display, forexample.

Accordingly drivers/owners of motor vehicles can pull or receive datafrom the insurance providers 4110, wherein received data can be posted(e.g., displayed on a monitor) and the connection instance can beconcurrently updated to reflect any successful and/or failed dataretrievals. Thus, at any given moment the connection instance caninclude the most up-to-date version of data transferred between themotor vehicle and the insurance providers. In an embodiment, a switchingcomponent 4806 can be configured to automatically switch user/driver toan insurance provider/company that bids the best rate. Such switchingcomponent 4806 can employ interrupts both in hardware and/or software toconclude the switching from one insurance provider to another insuranceprovider. For example, the interrupt can convey receipt of a moreoptimal insurance rate or completion of a pull request to the insuranceproviders 4110 or that a configuration has changed. In one particularaspect, once an interrupt occurs, an operating system analyzes the stateof the system and performs an action in accordance with the interrupt,such as a change of insurance provider, for example

Such interrupts can be in form of asynchronous external events to theprocessor that can alter normal program flow. Moreover, the interruptscan usually require immediate attention from a processor(s) associatedwith the system. In one aspect, when an interrupt is detected, thesystem often interrupts all processing to attend to the interrupt,wherein the system can further save state of the processor andinstruction pointers on related stacks.

According to a further aspect, the switching component 4804 can employan interrupt dispatch table in memory, which can be accessed by theprocessor to identify a function that is to be called in response to aparticular interrupt. For example, a function can accept a policy froman insurance provider, cancel an existing policy, and/or clear theinterrupt for a variety of other reasons. The function can executeprocesses such as clearing the state of the interrupt, calling a driverfunction to check the state of an insurance policy and clearing, settinga bit, and the like.

FIG. 19 is a diagram 4900 illustrates generally, the switching component806 that further includes an analyzer component 4902, which furtheremploys threshold ranges and/or value(s) (e.g., pricing ranges forinsurance policies, terms of the insurance policy, and the like)according to a further aspect of the present invention. The analyzercomponent 4902 can be configured to compare a received value forinsurance coverage to the predetermined thresholds, which can bedesignated by an owner/driver. Accordingly, the analyzer component 902can determine if the received insurance coverage policies are within thedesired range as specified by a user an “accept” or “reject”, and/orfurther create a hierarchy from “low” to “high” based on criteriadesignated by the user (e.g., price of the insurance policy, terms ofthe insurance policy, and the like).

According to a further aspect, the analyzer component 4902 can furtherinteract with a rule engine component 4904. For example, a rule can beapplied to define and/or implement a desired evaluation method for aninsurance policy. It is to be appreciated that the rule-basedimplementation can automatically and/or dynamically define and implementan evaluation scheme of the insurance policies provided. Accordingly,the rule-based implementation can evaluate an insurance policy byemploying a predefined and/or programmed rule(s) based upon any desiredcriteria (e.g., criteria affecting an insurance policy such as durationof the policy, number of drivers covered, type of risks covered, and thelike.).

In a related example, a user can establish a rule that can implement anevaluation based upon a preferred hierarchy (e.g., weight) of criteriathat affects the insurance policy. For example, the rule can beconstructed to evaluate the criteria based upon predeterminedthresholds, wherein if such criteria does not comply with setthresholds, the system can further evaluate another criteria orattribute(s) to validate the status (e.g., “accept” or “reject” theinsurance bid and operate the switching component based thereon). It isto be appreciated that any of the attributes utilized in accordance withthe subject invention can be programmed into a rule-based implementationscheme.

FIG. 20 illustrates generally, a method 5000 for customizing insurancerates of a driver, according to embodiments as described herein. Themethodology 5000 of customizing insurance rates according to a furtheraspect of the subject innovation. While the exemplary method isillustrated and described herein as a series of blocks representative ofvarious events and/or acts, the subject innovation is not limited by theillustrated ordering of such blocks. For instance, some acts or eventsmay occur in different orders and/or concurrently with other acts orevents, apart from the ordering illustrated herein, in accordance withthe innovation. In addition, not all illustrated blocks, events or acts,may be required to implement a methodology in accordance with thesubject innovation. Moreover, it will be appreciated that the exemplarymethod and other methods according to the innovation may be implementedin association with the method illustrated and described herein, as wellas in association with other systems and apparatus not illustrated ordescribed. Initially and at 5002 contextual data from various data bankscan be accessed by the insurance providers or supplied thereto. Asexplained earlier, the data banks can include data pertaining to themotor vehicle (e.g., maintenance history, current vehicle conditions,and the like), data related to the driver (e.g., via health insurancerecords, police records, internet records, and the like), and datarelated to operating environment (e.g., weather, geographical location,and the like.) Moreover, the real-time contextual driving data caninclude both an intensity portion and a frequency portion, whichrepresent severity and regularity of driving episodes (e.g., slammingthe brakes, gradual/sudden deceleration, velocity variances, and thelike). Subsequently and at 5004, such data can be analyzed by theinsurance providers as to customize an insurance rate based thereon at5006. In an embodiment, insurance rate can be calculated in real-timeand as such can more accurately reflect appropriate coverage for asituation of a driver. A plurality of different factors can influence alikelihood of the driver being involved in an accident, having a vehiclestolen, and the like. For example, if the driver is travelling throughbad weather, then risk can be higher and a rate can be increased inreal-time as weather conditions change-conversely, if there isrelatively little traffic surrounding the driver's vehicle, then therate can be lowered. An algorithm or complex model can be used tocalculate the insurance rates and can be disclosed to the driver throughthe display. In an embodiment, the rate adjustment component 804 can beconfigured to evaluate the insurance rate information against currentvehicle operation by the driver. Specifically, the evaluation cancompare the current operation against insurance rate information todetermine if an appropriate rate is being used, if the rate should bechanged, what the change should be, etc. For instance, different aspectsof vehicle operation can be taken into account such as for example, butnot limited to, weather and how a driver reacts, speed (of a vehicle),traffic and how the driver reacts, and noise {e.g., radio level), andthe like.

Subsequently, the customized insurance rate can then be sent from aninsurance provider to an owner/driver of the vehicle (e.g., in form ofan insurance bid) at 5008. For example, the insurance rate can bedetermined and represented upon the driver via the display or controllerin the vehicle. A processor that executes the computer executablecomponents stored on a storage medium can be employed. In an embodiment,the monitoring unit can communicate with an insurance company {e.g.,continuous communication) and obtain an insurance rate directly. Thesystem can be configured to customize the insurance based on theobtained insurance rates and present to the driver and make appropriatemodification to the display automatically.

FIG. 21 illustrates generally, a method 1100 for presenting informationrelated to a real-time insurance rate, according to embodiments asdescribed herein. In an embodiment, at 5102, Metadata can be collectedpertaining to real-time operation of a vehicle and at least a portion ofthe metadata can be evaluated, as shown at 5104. The metadata describedherein can include vehicle behavior data, contextual information, driverhistory, and real-time driving information that relates to operation ofa driver and vehicle, and the like. Based upon a result of theevaluation, there can be calculation a real-time insurance rate, such asshown at 5106. In an embodiment, at 5108, determination can be made onhow to present the calculated rate. For example, the determination canbe if the rate should be shown on a center console or a heads-updisplay. A determination can also be made on how to display data (e.g.,if a numerical rate should be disclosed or a color element should belit). Additionally, a determination can be made on other data todisclose, such as safety, environment impact, cost of operating vehicle,a target speed, group rank, and the like. The determined rate and otherdetermined data can be presented through a display, such as shown at5110. Thus, the determined rate is presented upon a display viewable tothe driver of the vehicle.

In an embodiment, at 5112, the method 5100 includes determining iffeedback should be presented to the user. The feedback can be suppliedin real-time as well as be a collective summary presented after adriving session is complete. If no feedback should be presented, thenthe method 5100 can end at 5114. In one instance, if there is a newdriver attempting to obtain a full drivers license (e.g., teenagedriver) or newer driver, then the check 5112 can determine feedbackshould be automatically provided. In another embodiment, an operator canbe solicited on if feedback should be presented depending on a responsethe method 5100 can end or continue.

Operation of the vehicle and driver can be evaluated at 5116, which canoccur though different embodiments. As a user operates a vehicle,metadata can be collected and evaluated in real-time. In an alternativeembodiment, data can be collected, but evaluation does not occur untilthe check 5112 determines feedback should be presented. At 5118, therecan be determining feedback for suggesting future driving actions forthe operator to perform in future driving to lower the insurance rate.The method 5100 can include presenting the feedback (e.g., through thedisplay, through a printout, transferring feedback as part of e-mail ora text message, etc.) at 5120. The feedback can be directly related to adriving session as well as is an aggregate analysis of overall drivingperformance (e.g., over multiple driving sessions).

FIG. 22 is diagram illustrates generally, a method 5200 for installationof a real-time insurance system, according to embodiments disclosedherein. In an embodiment, at 5202, an on-board monitoring system (suchas advertising monitoring unit) 4102 is installed in a vehicle tofacilitate the collection of real-time data from the vehicle andforwarding of the real-time data to an insurance provider. At 5204, theon-board monitoring system can be associated with the on-boarddata/diagnostic control units and system(s) incorporated into thevehicle. The on-board data/diagnostic control units and system(s) caninclude the vehicles engine control unit/module (ECU/ECM), transmissioncontrol unit (TCU), power train control unit (PCU), on-board diagnostics(OBD), sensors and processors associated with the transmission system,and other aspects of the vehicle allowing the on-board monitoring systemto gather sufficient data from the vehicle for a determination of howthe vehicle is being driven to be made. The on-board monitoring systemcan be communicatively coupled by hard wiring to the on-board diagnosticsystem(s) or the systems can be communicatively associated usingwireless technologies.

In an embodiment, at 5206, a mobile device (e.g., a cell phone) can beassociated with the onboard monitoring system where the mobile devicecan facilitate communication between the on-board monitoring systemswith a remote insurance provider system. The mobile device providesidentification information to the on-board monitoring system to beprocessed by the on-board monitoring system or forwarded an insuranceprovider system to enable identification of the driver.

In an embodiment, at 5208, communications are established between theon-board monitoring system and the mobile device with the remoteinsurance provider system. In one embodiment it is envisaged that theon-board monitoring system and the insurance provider system are ownedand operated by the same insurance company. However, the system could beless restricted whereby the insurance provider system is accessible by aplurality of insurance companies with the operator of the on-boardmonitoring system, e.g., the driver of the vehicle to which the on-boardmonitoring system is attached, choosing from the plurality of insuranceproviders available for their particular base coverage. In such anembodiment, upon startup of the system the insurance provider system candefault to the insurance company providing the base coverage and theoperator can select from other insurance companies as they require. Overtime, as usage of the on-board monitoring system continues, at 5210,there is a likelihood that various aspects of the system might need tobe updated or replaced, e.g., software update, hardware updates, etc.,where the updates might be required for an individual insurance companysystem or to allow the on-board monitoring system to function with oneor more other insurance company systems. Hardware updates may involvereplacement of a piece of hardware with another, while software updatescan be conducted by connecting the mobile device and/or the on-boardmonitoring system to the internet and downloading the software from acompany website hosted thereon. Alternatively, the software upgrade canbe transmitted to the mobile device or the on-board monitoring system bywireless means. As a further alternative the updates can be conferred tothe mobile device or the on-board monitoring system by means of aplug-in module or the like, which can be left attached to the respectivedevice or the software can be downloaded there from.

FIG. 23 is a diagram illustrates generally, a method for gatheringinformation from an on-board monitoring system employed in a real-timeinsurance system, according to embodiments as disclosed herein. In anembodiment, at 5302, monitoring of the driver and the vehicle they areoperating is commenced. Monitoring can employ components of an on-boardmonitoring system, mobile device components, e.g., cell phone system, orany other system components associated with monitoring the vehicle as itis being driven. Such components can include a global positioning system(GPS) to determine the location of the vehicle at any given time, such aGPS can be located in a cell phone, as part of the on-board monitoringsystem, or an external system coupled to the monitoring system/cellphone—such an external system being an OEM or after sales GPS associatedwith the vehicle to be/being driven. A video data stream can be gatheredfrom a video camera coupled to the on-board monitoring system recordingthe road conditions, etc. throughout the journey. Information can alsobe gathered from monitoring/control system(s) that are integral to thevehicle, e.g., the vehicle's engine control unit/module (ECU/ECM) thatmonitors various sensors located throughout the engine, fuel and exhaustsystems, etc.

In an embodiment, at 5304, the dynamically gathered data (or vehiclebehavior data) is transmitted to an insurance evaluation system. In anembodiment, at 5306, the gathered data is analyzed. Such analysis caninvolve identifying the route taken by the driver, the speed driven,time of day the journey was undertaken, weather conditions during thejourney, other road traffic, did the user use their cell phone duringthe journey?, and the like. In an embodiment, at 5308, the gathered datais assessed from which an insurance rate(s) can be determined. Forexample, if the driver drove above the speed limit then an appropriatedetermination could be to increase the insurance premium. In anembodiment, at 5310, the driver can be informed of the newly determinedinsurance rate. Any suitable device can be employed such as informingthe user by cell phone, a display device associated with the on-boardmonitoring system, or another device associated with the vehicle. Theinformation can be conveyed in a variety of ways, including a textmessage, a verbal message, graphical presentation, change of lightemitting diodes (LED's) on a display unit, a HUD, etc. At 5312, thedriver can continue to drive the vehicle whereby the method can returnto 5302 where the data gathering is commenced once more.

Alternatively, in an embodiment, at 5312, the driver may complete theirjourney and data gathering and analysis is completed. In an embodiment,at 5314 the driver can be presented with new insurance rates based uponthe data gathered while they were driving the vehicle. The new insurancerates can be delivered and presented to the driver by any suitablemeans, for example the new insurance rates and any pertinent informationcan be forwarded and presented to the driver via a HUD employed as partof the real time data gathering system. By employing a HUD instantaneousnotifications regarding a change in the driver's insurance policy can bepresented while mitigating driver distractions {e.g., line of sightremains substantially unchanged). Alternatively, the on-board monitoringsystem can be used, or a remote computer/presentation device coupled tothe real time data gathering system where the information is forwardedto the driver via, e.g., email. In another embodiment, the driver canaccess a website, hosted by a respective insurance company, where thedriver can view their respective rates/gathered information/analysissystem, etc. Further, traditional means of communication such as aletter can be used to forward the insurance information to the driver.

FIG. 24 is a diagram illustrates generally, a method 5400 mountingcameras to capture traffic information, according to embodiments asdisclosed herein. In an embodiment, at 5402, the method 5400 includesmounting cameras on the car to monitor the traffic information. Forexample, the car may include cameras mounted to capture views in therearward, downward, and the like directions, on the upper surface at theleading end of the front portion thereof. The position for mounting thecameras is not limited to the left side, right side, upper surface,front side, back side, and the like. For example, if the car has a leftside steering wheel, the camera may be mounted on a right upper surfaceat a leading end of the front portion of the car. The cameras may havean angle of view of about 60, 90, 180, and 360 degree. With theconstruction, since the camera is mounted for a view in the rearward anddownward directions on the front portion of the car, it can capture awide area of the surface of the road in the vicinity of the driver'scar, and an area in the vicinity of the left front wheel. Furthermore,the camera can also capture a part of the body of the car in thevicinity of the front wheel. Thereby, the relation between the car andthe surface of the road can be recorded. In an example, the cameras canbe configured to capture images of the road views including potentialcollision events such as how close car is following car in front, howoften brake is used in period of time, hard brakes count more to reducedriver rating, how frequently does car come close to objects andobstructions (such as trees, cars on the other direction and cars insame direction) while moving.

In an embodiment, at 5404, the method 5400 includes receiving therecorded information from the camera and use image processing techniquesto process the information. For example, the system uses imageprocessing techniques to determine potential collision events such ashow close car is following car in front, how often brake is used inperiod of time, hard brakes count more to reduce driver rating, howfrequently does car come close to objects and obstructions (such astrees, cars on the other direction and cars in same direction) whilemoving.

FIG. 25 is a diagram that illustrates generally, a method 5500 mountingcameras to capture vehicle behavior, according to embodiments asdisclosed herein. In an embodiment, at 5502, the method 5500 includesmounting cameras on the car to monitor the vehicle behavior. Theposition for mounting the cameras is not limited to the left side, rightside, upper surface, front side, back side, and the like. The camerasmay have an angle of view of about 60, 90, 180, and 360 degree. Forexample, the camera can capture vehicle behavior such as for example,but not limited to, images of texting and use of phone while driving,speech of driver shouting or cursing at other drivers or otheroccupants, indications of intoxication, sleepiness, alcohol level, mood,aggressiveness, and the like. In an embodiment, at 5504, the method 5500includes receiving the recorded information from the camera and useimage processing techniques and voice reorganization techniques toprocess the information. For example, the system uses image processingtechniques to determine the driver activity such as whether the driveris using mobile phone while driving. In another example, the system usesvoice recognition techniques to determine the use voice, text,aggressiveness, and the like.

In an embodiment, the item-centric approach determines that many drivershaving similar behavior and the driver who performs activity-A will alsoperform activity-B. This has proven to be fairly effective. On the otherhand, many insurance providers interact with drivers online/offline.Such interaction can produce a stream of contextual information thatrecommendation engines can use. Early systems were batch oriented andcomputed recommendations in advance for each driver. Thus, they couldnot always react to a driver's most recent behavior. Recommendationengines work by trying to establish a statistical relationship betweendrivers and activities associated with there behavior. The systemestablishes these relationships via information about driver's behaviorfrom vehicle owner, monitoring devices, sensors, and the like.

In an embodiment, the reasonableness determination systems collect datavia APIs, insurance application, insurance databases, and the likesources. The insurance sources can be available through social networks,ad hoc and marketing networks, and other external sources. For example,data can be obtained from insurance sites, insurance providers, driverinsurance history, and search engines. All this enables recommendationengines to take a more holistic view of the driver. The recommendationengine can recommend different insurance products that save money forthe driver, or alternatively can even recommend different insurancecompanies to save money. Using greater amounts of data lets the enginesfind connections that might otherwise go unnoticed, which yields bettersuggestions. This also sometimes requires recommendation systems to usecomplex big-data analysis techniques. Online public profiles andpreference listings on social networking sites such as Facebook adduseful data.

Most recommendation engines use complex algorithms to analyze vehiclebehavior and suggest recommended activities that employ personalizedcollaborative filtering, which use multiple agents or data sources toidentify behavior patterns and draw conclusions. This approach helpsdetermine that numerous drivers who have same or similar type ofbehavior in the past may have to perform one or more similar activitiesin the future. Many systems use expert adaptive approaches. Thesetechniques create new sets of suggestions, analyze their performance,and adjust the recommendation pattern for similar behavior of drivers.This lets systems adapt quickly to new trends and behaviors. Rules-basedsystems enable businesses to establish rules that optimizerecommendation performance.

FIG. 26 is a diagram 5600 illustrates generally, a first vehicle programcommunicating with a second vehicle program through an Inter-Vehiclenetworking, according to embodiments as disclosed herein. In anembodiment, the system develops inter-vehicular networking, computing,transceivers, and sensing technologies in the vehicles. Such vehicleshave embedded computers, GPS receivers, short-range wireless networkinterfaces, and potentially access to in-car sensors and the Internet.Furthermore, they can interact with road-side wireless sensor networksand sensors embedded in other vehicles. These capabilities can beleveraged into distributed computing and sensing applications overvehicular networks for safer driving, dynamic route planning, mobilesensing, or in-vehicle entertainment. The system can includevehicular-specific network protocols, middleware platforms, and securitymechanisms to process the data. As shown in FIG. 26, a first driveroperating a vehicle observes a second driver operating a vehicle withinhis visual range and wants to send a message to the second driver. Thevehicle can include identifying information that is visuallyascertainable such as the model, vehicle color, number of doors, licenseplate number and state. The vehicle may include additional informationthat is only ascertainable from up close or at certain angles, or viacertain technologies, such as a roof top identification number, vehicleidentification number, taxi badge number, Bluetooth, or RFID code, andthe like. In an embodiment, a sender having access to the vehiclemonitoring device and viewing a second vehicle desires to contact thedriver of the second vehicle. In one embodiment, the cars can cooperatespeed and display ads to optimize revenue. As best shown in FIG. 16, thesender initiates communication via a telephone or handheld computer orvehicle monitoring device and accesses the interface to theinter-vehicle networking service and database. The sender can select“send message” from the graphical or audio menu to send message ordirectly communicate with the driver of the second vehicle. For example,the sender can directly communicate with the driver using theinter-vehicle networking or the sender can choose from a table ofmessages that can be sent to the driver using the inter-vehiclenetworking. For example, the message can take the form of voice, audio,video, or other data which can be converted to a digital signal and sentto any communications terminal. The inter-vehicle networking databasereceives the message or encrypted message and reconstructs the message,including the ad data to be displayed. The inter-vehicle networking thencommands each vehicle to follow a planned route that meets the following2 objectives: get the occupant to destination meeting any timeconstraints, and maximize advertising profit by selecting apredetermined path through crowds with a predetermined level ofdisposable income.

In this case, the transceiver can be a WiMAX system. In anotherembodiment, the transceiver can be a meshed 802 protocol networkconfiguration with a constantly morphing mobile mesh network that helpsdrivers avoid accidents, identify traffic jams miles before theyencounter them, and act as a relay point for Internet access. In oneembodiment, the mesh network can be the ZigBee mesh network. In anotherembodiment, the mesh network can be a modified Wi-Fi protocol called802.11p standard for allowing data exchange between moving vehicles inthe 5.9 GHz band. 802.11p operates in the 5.835-5.925 GHz range, dividedinto 7 channels of 10 MHz each. The standard defines mechanisms thatallow IEEE 802.11™ technology to be used in high speed radioenvironments typical of cars and trucks. In these environments, the802.11p enhancements to the previous standards enable robust andreliable car-to-car and car-to-curb communications by addressingchallenges such as extreme Doppler shifts, rapidly changing multipathconditions, and the need to quickly establish a link and exchange datain very short times (less than 100 ms). Further enhancements are definedto support other higher layer protocols that are designed for thevehicular environment, such as the set of IEEE 1609™ standards forWireless Access in Vehicular Environments (WAVE). 802.11p supportsIntelligent Transportation Systems (ITS) applications such ascooperative safety, traffic and accident control, intersection collisionavoidance, and emergency warning.

One variation of 802.11p is called the Dedicated Short RangeCommunications (DSRC), a U.S. Department of Transportation project aswell as the name of the 5.9 GHz frequency band allocated for the ITScommunications. More information on the 802.11p standard can be obtainedfrom the IEEE. DSRC itself is not a mesh. It's a broadcast, so it onlyreaches vehicles within range. Meshing requires a lot moresophistication. There's a routing aspect to it, relaying messages toother nodes. DSRC is much simpler.

One embodiment uses high-powered, heavily encrypted Wi-Fi thatestablishes point-to-point connections between cars within a half-mileradius. Those connections are used to communicate vital informationbetween vehicles, either triggering alerts to the driver or interpretedby the vehicle's computer. An intelligent car slamming on its brakescould communicate to all of the vehicles behind it that it's coming torapid halt, giving the driver that much more warning that he too needsto hit the brakes.

But because these cars are networked—the car in front of one vehicle isconnected to the car in front it and so forth—in a distributed mesh, anintelligent vehicle can know if cars miles down the road are slamming ontheir brakes, alerting the driver to potential traffic jams. Givenenough vehicles with the technology, individual cars become nodes in aconstantly changing, self-aware network that can not only monitor what'sgoing on in the immediate vicinity, but across a citywide traffic grid.

In one embodiment, the processor receives travel routes and sensor datafrom adjacent vehicles, such information is then used for preparingvehicular brakes for a detected turn or an anticipated turn fromadjacent vehicles. The travel routes can be transmitted over a vehicularWi-Fi system that sends protected information to nearby vehiclesequipped with Wi-Fi or Bluetooth or ZigBee nodes. In one embodiment, amesh-network is formed with Wi-Fi transceivers, wherein each vehicle isgiven a temporary ID in each vehicular block, similar to a cellularblock where vehicles can join or leave the vehicular block. Once thevehicle joins a group, travel routes and sensor data is transferredamong vehicles in a group. Once travel routes are shared, the processorcan determine potential or desired actions from the adjacent vehiclesand adjust appropriately. For example, if the car in front of thevehicle is about to make a turn, the system prepares the brakes andgently tugs the driver's seat belt to give the drive notice that the carin front is about to slow down. In another example, if the processordetects that the driver is about to make a lane change to the left basedon sensor data and acceleration pedal actuation, but if the processordetects that the vehicle behind in the desired lane is also speeding up,the system can warn the driver and disengage the lane change to avoidthe accident. Thus, the processor receives travel routes and sensor datafrom adjacent vehicles and notifying the driver of a detected turn or ananticipated turn from adjacent vehicles. The processor receives travelroutes and sensor data from adjacent vehicles and optimizes groupvehicular speed to improve fuel efficiency. The processor receivestravel routes and sensor data from adjacent vehicles and sequences redlight(s) to optimize fuel efficiency. The processor notifies the driverof driving behaviors from other drivers at a predetermined location. Theprocessor switches turn signals and brakes using a predeterminedprotocol to reduce insurance premium for the driver. The processor warnsthe driver to avoid driving in a predetermined pattern, driving during apredetermined time, driving in a predetermined area, or parking in apredetermined area to reduce insurance premium for the driver. Theprocessor sends vehicle behavior data to an insurer, including at leastone of: vehicle speed, vehicle accelerations, vehicle location, seatbeltuse, wireless device use, turn signal use, detection of ethanol vapor,driver seating position, and time.

The various systems described above may be used by the computer tooperate the vehicle and maneuver from one location to another. Forexample, a user may enter destination information into the navigationsystem, either manually or audibly. The vehicle may determine itslocation to a few inches based on a combination of the GPS receiverdata, the sensor data, as well as the detailed map information. Inresponse, the navigation system may generate a route between the presentlocation of the vehicle and the destination.

When the driver is ready to relinquish some level of control to theautonomous driving computer, the user may activate the computer. Thecomputer may be activated, for example, by pressing a button or bymanipulating a lever such as gear shifter. Rather than taking controlimmediately, the computer may scan the surroundings and determinewhether there are any obstacles or objects in the immediate vicinitywhich may prohibit or reduce the ability of the vehicle to avoid acollision. In this regard, the computer may require that the drivercontinue controlling the vehicle manually or with some level of control(such as the steering or acceleration) before entering into a fullyautonomous mode.

Once the vehicle is able to maneuver safely without the assistance ofthe driver, the vehicle may become fully autonomous and continue to thedestination. The driver may continue to assist the vehicle bycontrolling, for example, steering or whether the vehicle changes lanes,or the driver may take control of the vehicle immediately in the eventof an emergency.

The vehicle may continuously use the sensor data to identify objects,such as traffic signals, people, other vehicles, and other objects, inorder to maneuver the vehicle to the destination and reduce thelikelihood of a collision. The vehicle may use the map data to determinewhere traffic signals or other objects should appear and take actions,for example, by signaling turns or changing lanes. Once the vehicle hasarrived at the destination, the vehicle may provide audible or visualcues to the driver. For example, by displaying “You have arrived” on oneor more of the electronic displays.

The vehicle may be only partially autonomous. For example, the drivermay select to control one or more of the following: steering,acceleration, braking, and emergency braking.

The vehicle may also have one or more user interfaces that allow thedriver to reflect the driver's driving a style. For example, the vehiclemay include a dial which controls the level of risk or aggressivenesswith which a driver would like the computer to use when controlling thevehicle. For example, a more aggressive driver may want to change lanesmore often to pass cars, drive in the left lane on a highway, maneuverthe vehicle closer to the surrounding vehicles, and drive faster thanless aggressive drivers. A less aggressive driver may prefer for thevehicle to take more conservative actions, such as somewhat at or belowthe speed limit, avoiding congested highways, or avoiding populatedareas in order to increase the level of safety. By manipulating thedial, the thresholds used by the computer to calculate whether to passanother car, drive closer to other vehicles, increase speed and the likemay change. In other words, changing the dial may affect a number ofdifferent settings used by the computer during its decision makingprocesses. A driver may also be permitted, via the user interface, tochange individual settings that relate to the driver's preferences. Inone embodiment, insurance rates for the driver or vehicle may be basedon the style of the driving selected by the driver.

Aggressiveness settings may also be modified to reflect the type ofvehicle and its passengers and cargo. For example, if an autonomoustruck is transporting dangerous cargo (e.g., chemicals or flammableliquids), its aggressiveness settings may be less aggressive than a carcarrying a single driver—even if the aggressive dials of both such atruck and car are set to “high.” Moreover, trucks traveling across longdistances over narrow, unpaved, rugged or icy terrain or vehicles may beplaced in a more conservative mode in order reduce the likelihood of acollision or other incident.

In another example, the vehicle may include sport and non-sport modeswhich the user may select or deselect in order to change theaggressiveness of the ride. By way of example, while in “sport mode”,the vehicle may navigate through turns at the maximum speed that issafe, whereas in “non-sport mode”, the vehicle may navigate throughturns at the maximum speed which results in g-forces that are relativelyimperceptible by the passengers in the car.

The vehicle's characteristics may also be adjusted based on whether thedriver or the computer is in control of the vehicle. For example, when aperson is driving manually the suspension may be made fairly stiff sothat the person may “feel” the road and thus drive more responsively orcomfortably, while, when the computer is driving, the suspension may bemade such softer so as to save energy and make for a more comfortableride for passengers.

For purposes of illustration, a number of example implementations aredescribed. It is to be understood, however, that the exampleimplementations are illustrative only and are not meant to limiting.Other example implementations are possible as well.

As used herein, the term “information” includes communication and/orreception of knowledge and/or intelligence, and knowledge obtained frominvestigation, study, and/or instruction. Information also includesdata, such as information in numerical form that can be digitallytransmitted and/or processed. Thus, information includes raw data andunorganized facts that can be processed, and also includes processed,organized, and structured, such as data being processed and presented ina useful context.

As used herein, “private information” includes information pertaining toa user's behavior, actions, and/or communication that occurs in acontext in which an individual can reasonably expect privacy,information provided by an individual for a specific purpose and forwhich the individual can reasonably expect will not be made public (forexample, an email between two friends or medical records), and/orinformation that is not known or intended to be known publicly. Privateinformation can be known by a single person (such as a user knowing asecret about himself or herself), or it can be known to more than oneperson and still be private (such as personal information about a userthat is known to the user, the user's family, and the user's friends butnot known to the public or not readily accessible to the public).

As used herein, a “product” is something produced by human effort,mechanical effort, and/or a natural process. The term product can alsoinclude a service that is an act or work performed for a user and forpay.

As used herein, a “social network” is a social structure in which userscommunicate with each other over a network with electronic devices. Thesocial network facilitates the building of social relations among userswho share backgrounds, familial relations, business relations,interests, and/or connections. The social network includes one or moreof representations and/or information about the users (such as userprofiles, photos, videos, etc.) and a platform (such as a web-basedplatform) that allows the users to communicate with each other over oneor more networks (such as using email and/or instant messages over theInternet) and/or share information with other users in the socialnetwork.

As used herein, “target advertising” and “target advertisement” and“target video advertisement” are advertising in which advertisements aredirected to a consumer based on personal traits or personal preferencesof the consumer, such as demographics, psychographics (personality,values, attitudes, interests, and lifestyles), purchase history, and/oruser profile.

As used herein, a “user” is a human being, a person.

As used herein, a “user profile” is personal data that represents anidentity of a specific person or organization. The user profile includesinformation pertaining to the characteristics and/or preferences of theuser. Examples of this information for a person include, but are notlimited to, one or more of personal data of the user (such as age,gender, race, ethnicity, religion, hobbies, interests, income,employment, education, etc.), photographs (such as photos of the user,family, friends, and/or colleagues), videos (such as videos of the user,family, friends, and/or colleagues), and user-specific data that definesthe user's interaction with and/or content on an electronic device (suchas display settings, application settings, network settings, storedfiles, downloads/uploads, browser activity, software applications, userinterface or GUI activities, and/or privileges).

In some example embodiments, the methods illustrated herein and data andinstructions associated therewith are stored in respective storagedevices, which are implemented as computer-readable and/ormachine-readable storage media, physical or tangible media, and/ornon-transitory storage media. These storage media include differentforms of memory including semiconductor memory devices such as DRAM, orSRAM, Erasable and Programmable Read-Only Memories (EPROMs),Electrically Erasable and Programmable Read-Only Memories (EEPROMs) andflash memories; magnetic disks such as fixed, floppy and removabledisks; other magnetic media including tape; optical media such asCompact Disks (CDs) or Digital Versatile Disks (DVDs). Note that theinstructions of the software discussed above can be provided oncomputer-readable or machine-readable storage medium, or alternatively,can be provided on multiple computer-readable or machine-readablestorage media distributed in a large system having possibly pluralnodes. Such computer-readable or machine-readable medium or media is(are) considered to be part of an article (or article of manufacture).An article or article of manufacture can refer to any manufacturedsingle component or multiple components.

Method blocks discussed herein can be automated and executed by acomputer, computer system, user agent, and/or electronic device. Theterm “automated” means controlled operation of an apparatus, system,and/or process using computers and/or mechanical/electrical deviceswithout the necessity of human intervention, observation, effort, and/ordecision.

The methods in accordance with example embodiments are provided asexamples, and examples from one method should not be construed to limitexamples from another method. Further, methods discussed withindifferent figures can be added to or exchanged with methods in otherfigures. Further yet, specific numerical data values (such as specificquantities, numbers, categories, etc.) or other specific informationshould be interpreted as illustrative for discussing exampleembodiments. Such specific information is not provided to limit exampleembodiments. It should be understood, of course, that the foregoingrelates to exemplary embodiments of the invention and that modificationsmay be made without departing from the spirit and scope of the inventionas set forth in the following claims.

What is claimed is:
 1. A vehicle, comprising: a ridesharing fareindicator; a display mounted on a roof to show advertisements, thedisplay including a cellular transceiver in communication with a remoteserver and a processor controlling the display based on one or more ofthe following: advertising parameters from the advertiser; mobilelocation; mobile distance from a landmark; advertising categories;mobile location demographics; pricing of advertised goods or services;time and date; speed and direction of the advertising display; trafficcharacteristics associated with the display; views of the display andadvertising budget characteristics, wherein the display operates inconcert with one or more additional vehicles operating as a group toshow a predetermined sequence of advertisements upon entry to alocation.
 2. The vehicle of claim 1, comprising a vehicle camera,wherein the processor retrieves camera data to determine a number ofviews of the display.
 3. The vehicle of claim 1, wherein the displaycomprises one or more cameras and eye detection to determine a number ofviews of an advertisement shown on the display.
 4. An advertisingsystem, comprising: a two-wheeled vehicle; a display mounted on thevehicle to show advertisements; and a processor coupled to the display,wherein the processor captures data associated with showing anadvertisement on the display, wherein the data includes one or more ofthe following: advertising parameters from the advertiser; mobilelocation demographics; pricing of advertised goods or services; time anddate; speed and direction of the advertising display; trafficcharacteristics associated with the display; views of the display andadvertising budget characteristics; wherein the processor operates inconcert with one or more additional vehicles operating as a group toshow a predetermined sequence of advertisements upon entry to alocation.
 5. The vehicle of claim 1, wherein the processor communicateswith the remote server to send a payment or a reward to a vehicle driveror a vehicle owner based on one or more of the following: advertisingparameters from the advertiser; mobile location; mobile distance from alandmark; advertising categories; mobile location demographics; pricingof advertised goods or services; time and date; speed and direction ofthe advertising display; traffic characteristics associated with thedisplay; views of the display; and advertising budget characteristics.6. The vehicle of claim 1, wherein the processor controls a fullyautonomous or self-driving vehicle.
 7. The vehicle of claim 1,comprising a cabin camera, wherein the processor processes cabin cameraimages to detect emotion, drowsiness or fatigue from combination ofdetermining facial expression, hand gesture, and heart rate or breathingrate.
 8. The vehicle of claim 1, wherein the display communicates withnearby people through audio or visual responses.
 9. The vehicle of claim1, wherein the display is mounted on a roof, a side panel, front panel,or a rear panel of The vehicle.
 10. The vehicle of claim 1, wherein avehicle path is optimized for advertisement earnings.
 11. The system ofclaim 1, comprising a ridesharing server communicating with theprocessor to render a vehicle identification message to a passenger. 12.The system of claim 1, comprising one or more additional vehiclesoperating as a flock to show a sequence of related advertisements,wherein the additional vehicles include two wheeled, three wheeled, orat least four wheeled vehicles.
 13. The system of claim 1, comprising:a. a vehicle to consumer transceiver to poll consumers near a vehicle;and b. server coupled to The vehicle and to a credit card merchantdatabase to correlate purchase pattern with an advertisement shown to aconsumer exposed to the advertisement by The vehicle.
 14. The vehicle ofclaim 4, wherein the processor captures data associated with showing anadvertisement on the display.
 15. The system of claim 4, comprising acamera to determine a number of views of the display.
 16. The system ofclaim 4, wherein the display comprises one or more cameras and whereinthe processor performs eye detection to determine a number of views ofan advertisement shown on the display.
 17. The system of claim 4,wherein the processor communicates with a remote processor to send apayment or a reward to a vehicle driver or a vehicle owner based on oneor more of the following: advertising parameters from the advertiser;mobile location; mobile distance from a landmark; advertisingcategories; mobile location demographics; pricing of advertised goods orservices; time and date; speed and direction of the advertising display;traffic characteristics associated with the display; views of thedisplay; and advertising budget characteristics.
 18. The system of claim4, wherein the display is outside of a delivery container.
 19. Thesystem of claim 4, wherein the processor processes camera images todetect emotion, drowsiness or fatigue from combination of determiningfacial expression, hand gesture, and heart rate or breathing rate. 20.The system of claim 4, wherein the processor communicates with nearbypeople through audio or visual responses.